Harvest Portal API

This page describes how to search datasets, retrieve file details, and extract data directly from spreadsheet-format files through the Harvest Portal API. Write, account, and administrative actions are intentionally excluded from this documentation. For copy-paste JavaScript, Python, and cURL code samples, see the API Examples page.

The Harvest Portal is built on an open-source data catalog platform called CKAN. For the complete technical reference covering everything the underlying platform supports, including write operations not covered here, consult the official CKAN API Guide and DataStore API reference.

Searchable Files and Download-Only Files

Each dataset consists of one or more files, referred to on the Harvest Portal as "resources." Most resources are available only as direct downloads. When a resource is a spreadsheet-format file, however, Harvest Portal automatically loads it into a searchable table referred to as the "DataStore." Once a resource has been loaded into the DataStore, it can be searched, filtered, and queried for specific rows and columns through the API, without downloading the entire file.

On the Harvest Portal, this automatic processing currently applies to the following spreadsheet formats:

To determine whether a given resource has been processed in this way, view its page on the site; a processed resource displays a data preview table. Programmatically, this can be confirmed by checking the datastore_active field returned by the resource_show action (see the API Examples page for code samples).

All other formats, including Parquet, GeoTIFF, shapefiles, and JSON, are not processed into the DataStore and are available only as direct downloads through this API. Such files can still be queried efficiently using other tools; the API Examples page includes a Parquet example demonstrating this.

Authentication and Private Datasets

Requests made without credentials return only public datasets. To access datasets belonging to organizations associated with your account, including private datasets, generate an API token from your profile under Manage → API Tokens, and include it in the Authorization header of each request. This token should be kept confidential, as anyone in possession of it can act on behalf of your account.

Code Examples

The API Examples page has copy-paste cURL, JavaScript, and Python examples covering dataset search, DataStore queries, direct file downloads, and querying Parquet files without loading them into the DataStore.