ckan / datapusher

A standalone web service that pushes data files from a CKAN site resources into its DataStore
GNU Affero General Public License v3.0
77 stars 154 forks source link

Does not support resource with FTP URL-s #188

Open Chealer opened 4 years ago

Chealer commented 4 years ago

Disclaimer: I did not verify this issue (I assume a change equivalent to one made for pip had a result equivalent to the one for pip).

DataPusher 0.0.6 supported loading resources over FTP. However, this seems to be no longer the case starting from 0.0.13, after commit 461019713b2e48e20b32df4fe37ac5907cc07edf, which stopped DataPusher from using urllib2 to use the "Requests: HTTP for Humans" library, which - unsurprisingly - only supports HTTP, as it is based on urllib3, which - despite its name - does not support FTP like urllib2 did.

I'm filing this to document the change but also so that the code can be cleaned up. The error message for unsupported schemes ("Only http, https, and ftp resources may be fetched.") has not changed and suggests that FTP resources are still supported.

Note that DataPusher's alternative xloader does not support FTP any more.