ckan / datapusher

A standalone web service that pushes data files from a CKAN site resources into its DataStore
GNU Affero General Public License v3.0
76 stars 152 forks source link

Datapusher fails when trying to push json and pdf formats to the datastore #265

Open tazchalmers opened 3 months ago

tazchalmers commented 3 months ago

Datapusher using Alpine based Docker image ckan/ckan-base-datapusher:0.0.20 fails when trying to push json and pdf (and other) formats.

I do note that "DataPusher is a standalone web service that automatically downloads any tabular data files like CSV or Excel from a CKAN site's resources when they are added to the CKAN site, parses them to pull out the actual data, then uses the DataStore API to push the data into the CKAN site's DataStore."

However, the button allows me to click "Upload to Datastore" and the problem is that it fails in a horrible way for the user with an unfriendly Python traceback in the UI. Maybe it could be made not possible to click the "Upload to Datastore" button for certain formats and/or present the user with more of a "This format cannot be pushed to the DataStore" kind of message?

(I have edited this since I first posted as I realised that the datapusher did not automatically try and push to the datastore).