search

ThreeSixtyGiving / data-conversion

Documentation on data sources that are converted to the 360 Giving Standard
Apache License 2.0
1 stars 0 forks source link

dkan_calls.py seems to be broken #14

Open caprenter opened 9 years ago

caprenter commented 9 years ago

https://github.com/ThreeSixtyGiving/data-conversion/blob/master/scripts/dkan_calls.py

Is returning 404 errors.

I think https://github.com/ThreeSixtyGiving/data-conversion/blob/master/scripts/dkan_calls.py#L45 is wrong and should be

url = base_url + "group_package_show?id=" + result

but I can't see why that should be failing now, and not before?

However if I do make that change I then get errors: File "dkan_calls.py", line 52, in for resource in data_package['result']['resources']: TypeError: list indices must be integers, not str

I'm just stuck!

caprenter commented 9 years ago

I think I know what's going on:

At the DKAN, there has been a tidy up through the admin interface, so now many of the 'datasets' have the same name as the 'groups'. This means that (for some reason) a call to 'package_show' with an id that is the same as a group, returns a 404.

I'm guessing packages should not really have the same name as groups.

Changing the line to group_package_show is fundamentally a different call (getting the packages for a group) The results of 'package_list' should not be fed into group_package_show (but the results from group_list could be)

Feeding package ids (which are the same as group ids) into group_package_show will return data BUT that data is returned as a list, where as package_show returns a dictionary - hence the problem parsing the data.

The solution here is to make a DKAN call to get all groups and then to get the datasets from that.

We are working round problems here - so this assumes that all data on the DKAN is assigned to a group...