Harvest Portal API Examples

This page collects copy-paste examples for common tasks. For an explanation of the underlying concepts - the DataStore, which file formats support it, and authentication - see the Harvest Portal API documentation. Replace RESOURCE_ID, DATASET_ID, and YOUR_API_TOKEN with real values throughout.

cURL

Quick reference for testing from a terminal.

# Search public datasets
curl "https://data.harvestportal.org/api/3/action/package_search?q=maize&rows=5"

# Search including datasets your account can access
curl "https://data.harvestportal.org/api/3/action/package_search?rows=5" \
  -H "Authorization: YOUR_API_TOKEN"

# Get a single dataset's full metadata, including its resources
curl "https://data.harvestportal.org/api/3/action/package_show?id=DATASET_ID"

# Query rows from a resource loaded into the DataStore, filtered to an exact value
curl -X POST "https://data.harvestportal.org/api/3/action/datastore_search" \
  -H "Content-Type: application/json" \
  -d '{"resource_id": "RESOURCE_ID", "filters": {"country": "Kenya"}, "limit": 10}'

JavaScript

Search datasets

// Public datasets - no token needed
const res = await fetch(
  'https://data.harvestportal.org/api/3/action/package_search?q=maize&rows=5'
);
const { result } = await res.json();
console.log(result.results);

Get a single dataset’s metadata and resources

const res = await fetch(
  'https://data.harvestportal.org/api/3/action/package_show?id=DATASET_ID'
);
const { result: dataset } = await res.json();
console.log(dataset.resources);

Query rows from a resource in the DataStore

const res = await fetch(
  'https://data.harvestportal.org/api/3/action/datastore_search',
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: 'YOUR_API_TOKEN' // omit for public-only access
    },
    body: JSON.stringify({
      resource_id: 'RESOURCE_ID',
      filters: { country: 'Kenya' },
      limit: 10
    })
  }
);
const { result } = await res.json();
console.log(result.records);

Download a resource file directly

const res = await fetch(
  'https://data.harvestportal.org/dataset/DATASET_ID/resource/RESOURCE_ID/download/FILENAME'
);
const blob = await res.blob();

Query a Parquet resource directly in the browser

For file formats the DataStore does not support, such as Parquet, a Parquet-aware library can read the file directly using the same download URL. For example, with hyparquet(opens in a new tab):

const { asyncBufferFromUrl, parquetReadObjects } =
  await import('https://cdn.jsdelivr.net/npm/hyparquet/src/hyparquet.min.js');

const url = 'https://data.harvestportal.org/dataset/DATASET_ID/resource/RESOURCE_ID/download/FILENAME';
const file = await asyncBufferFromUrl({ url });
const rows = await parquetReadObjects({ file });
console.log(rows.slice(0, 10));

Python (ckanapi)

Search datasets

from ckanapi import RemoteCKAN

# Public datasets - no token needed
ckan = RemoteCKAN('https://data.harvestportal.org')
results = ckan.action.package_search(q='maize', rows=5)

Get a single dataset’s metadata and resources

dataset = ckan.action.package_show(id='DATASET_ID')
for resource in dataset['resources']:
    print(resource['id'], resource['name'], resource['datastore_active'])

Query rows from a resource in the DataStore

# With a token, to also see datasets your account can access
ckan = RemoteCKAN('https://data.harvestportal.org', apikey='YOUR_API_TOKEN')
result = ckan.action.datastore_search(
    resource_id='RESOURCE_ID', filters={'country': 'Kenya'}, limit=10
)
print(result['records'])

Download a resource file directly

import requests

resource = ckan.action.resource_show(id='RESOURCE_ID')
response = requests.get(resource['url'])
with open('downloaded_file', 'wb') as f:
    f.write(response.content)

Query a Parquet resource without downloading it first

Loading a columnar format such as Parquet into the DataStore would negate its principal advantages. Instead, a Parquet-aware tool can read the resource’s download URL directly. DuckDB, for example, can query a Parquet file over HTTP using range requests, without downloading the file in its entirety:

import duckdb

resource = ckan.action.resource_show(id='RESOURCE_ID')
duckdb.sql(f"SELECT * FROM read_parquet('{resource['url']}') LIMIT 10").show()

pandas with pyarrow, and Polars, are also capable of reading the same URL directly.