search

ckan / ckanapi

A command line interface and Python module for accessing the CKAN Action API
Other
176 stars 74 forks source link

loads datasets upload gibberish #176

Open frafra opened 3 years ago

frafra commented 3 years ago

Apparently random data are uploaded (strange symbols and such). Could it be an encoding error? I am using Python 3.8.

frafra commented 3 years ago

It seems to return a gzip file containing an HTML response "Resource not found".

frafra commented 3 years ago

It looks for an url in the JSON file, which is not there, as ckanapi dumps datasets creates resources like this:

    {
      "description": "Coordinates of all sites included in the dataset",
      "format": "TXT",
      "name": "s-ptarmigan-counts-coordinates-txt",
      "path": "data/s-ptarmigan-counts-coordinates-.txt",
      "title": "S_ptarmigan_counts_coordinates.txt"
    },

https://github.com/ckan/ckanapi/blob/8c563b859a92ac0fc97af6ae9cf79108f01043ac/ckanapi/cli/load.py#L289

frafra commented 3 years ago

In addition to making dump compatible with loads, a suggestion could be to raise a warning or an error when accessing a resource with an HTTP error.

frafra commented 2 years ago

ckanapi load datasets fails even when using ckanapi action ckan_package_search output, because it cannot get private datasets. CKAN returns a 404, potentially gzipped, which then gets uploaded as resource.