search

okfn / ckanclient-deprecated

DEPRECATED - please see https://github.com/ckan/ckanapi. [Python client library for CKAN]
http://pypi.python.org/pypi/ckanclient
25 stars 17 forks source link

Use python-requests to upload file #33

Closed Hoedic closed 10 years ago

Hoedic commented 10 years ago

Use python-requests (instead of pycurl) to upload file to CKAN.

rufuspollock commented 10 years ago

@davidread are you happy to review this?

davidread commented 10 years ago

Thanks Hoedic!!

In testing this with both datahub.io and demo.ckan.org, I get an error back from Google Storage - see below. I've no idea whether this is a problem with the client or whether CKAN is pointing the client at storage which is not configured.

>>> import ckanclient
>>> ckan = ckanclient.CkanClient(base_location='http://datahub.io/api', api_key='<key>')
>>> ckan.upload_file('/vagrant/test.csv')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/vagrant/ckanclient/ckanclient/__init__.py", line 582, in upload_file
    raise CkanApiError(body)
ckanclient.CkanApiError: <?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidArgument</Code><Message>Invalid argument.</Message><Details>Cannot create buckets using a POST.</Details></Error>

@Hoedic which server did you test on? @rgrp do you know about this Google Storage error?

I've put the merged version that I tested on the branch: okfn/pull-33-requests

Hoedic commented 10 years ago

Hey @davidread ,

I tested on CKAN 2.1 with a local storage. the error message sounds like a google storage issue but this is rather strange.

Did you try to upload the file using the last commit before I start playing with Ckanclient?

rufuspollock commented 10 years ago

@Hoedic I think there were perhaps some issues earlier with the upload - see #34 Right now I think we merge this and then fix in master and then release :-)

davidread commented 10 years ago

@Hoedic good idea to try the old version. I tried 183ad055e5 (June 2012) and it works fine:

>>> ckan.upload_file('/vagrant/test.csv')
(u'https://ckannet-storage.commondatastorage.googleapis.com/2014-01-09T143728/test.csv', '')

so the storage is working fine. However your code using requests still suffers the same error about creating a bucket with a POST. I also managed to test the current master using pycurl and it too has the same problem.

Since it is no worse than master, I'm going to merge it and raise an issue, which I'd be pleased if you were able to look at. I think this needs sorting before we release.