search

BergWerkGIS / QGIS-CKAN-Browser

Download and display Open Data from CKAN Metadata Servers
MIT License
34 stars 14 forks source link

Add filters for file/service types #4

Open ndkv opened 8 years ago

ndkv commented 8 years ago

In addition to geo data, many CKAN catalogues contain non-geo data as well. Showing the latter in QGIS does not make much sense.

It would be nice therefore to be able to filter on common geo file/service types such as Shapefiles, GeoJSON, WMS, WFS, etc.

Can you indicate which files in the codebase need to be extended in order to implement this?

wilhelmberg commented 8 years ago

Probably around here, if the CKAN API provides filtering on file type of resources. Does it?

Currently we use resources[].format to display the format:

image

Unfortunately not all servers provide that. So filtering would work only with those that provide a format.

Any ideas?

ndkv commented 8 years ago

I poked around the CKAN API and found /resource_search that takes a query parameter like so

https://data.overheid.nl/data/api/action/resource_search?query=format:JSON

This returns all JSON resources. It also works with SHAPE and WFS

https://data.overheid.nl/data/api/action/resource_search?query=format:SHAPE https://data.overheid.nl/data/api/action/resource_search?query=format:wfs

The API returns more resources than are listed in the web interface, though. I guess it also fetches unpublished ones?

Of course, this won't work if servers don't provide a value for format. Do you have an example of one?

wilhelmberg commented 8 years ago

resource_search?query=format:JSON

Great find.

Could be implemented with doing a resource_search first for getting the package ids and then doing a package_search with the ids as query filter. Although the query URL might get too long.

The API returns more resources than are listed in the web interface, though. I guess it also fetches unpublished ones?

Don't know for sure, but as I see it https://data.overheid.nl/data/api/action/resource_search?query=format:SHAPE returns resources whereas https://data.overheid.nl/data/dataset?res_format=SHAPE returns datasets.

Maybe there are datasets with more than one shape? Like this one: https://data.overheid.nl/data/dataset/bomen-met-eetbare-vruchten-gemeente-amersfoort

Of course, this won't work if servers don't provide a value for format. Do you have an example of one?

We tested several dozens of servers, so I don't remember by heart and I couldn't find one after a quick look into my notes. But I'm sure there were some, because of https://github.com/BergWerkGIS/QGIS-CKAN-Browser/blob/v0.2.8/CKAN-Browser/ckan_browser_dialog.py#L242-L244

One that at least comes close by providing a key with an empty value (format: ""): https://data.overheid.nl/data/api/3/action/package_show?id=4b2fb7c2-65c2-4cac-858a-a2525725134e

ndkv commented 8 years ago

You are completely right about resources vs. datasets: datasets can contain many resources, see the docs.

You could decide to ignore "formatless" resources as most of them will be useless (to QGIS). The one you link to has two resources. The "formatless" one eventually leads to a 404.