Add georeferencing widgets: draw dataset extent on map, or select pre-defined extent

[florianm]

This feature request concerns ckanext-spatial, ckanext-scheming and maybe it's not CKAN-core enough to get traction here - please advise if this is the wrong place to ask.

In the spatial widget, let users either pick a pre-defined spatial extent (e.g. administrative boundaries), or draw the extent directly (as rectangle or polygon), in addition to the good old paste-valid-GeoJSON which non-spatial users tend to struggle with.

Here's a tentative implementation in action: https://vimeo.com/116324887

If you'd like to have a go for yourself, request a login to http://data-demo.dpaw.wa.gov.au/

See also https://github.com/ckan/ckanext-spatial/pull/93 and https://github.com/open-data/ckanext-scheming/issues/11 (with backstory and use cases).

[jqnatividad]

Awesome stuff @florianm!

Another way to implement this is to use PostGIS to automatically compute the spatial extent after pushing data to the datastore for datasets with latitude/longitude columns using ST_Extent for bounding box and/or ST_ConcaveHull for "shrink-wrapped" polygons and automagically populate the spatial metadata.

We actually implemented automatically computing spatial-extents with our CartoDB integration.

[florianm]

Thanks @jqnatividad! Direct harvesting of a bounding box from data is extremely useful and should be another option.

We decided we need a manual option as sometimes we needed the extent to be known before data was available, or the dataset pertains to a well-defined area (e.g. a marine protected area) and needs to be discoverable via spatial search over that whole area.

Also, that CartoDB integration looks stunning. Need to give it a whirl right away!

[acheype]

Thanks @florianm for this really interesting job.

Is there any news about a merge of your work with ckan/ckanext-spatial and ckan/ckanext-scheming ? It seams unbelievable for me not to have by default an option to fill spatial extends with an appropriate widget !!

I'm interesting to install one ckan based on these features, it's a non-negotiable requirement for the scientist if I want to deploy an CKAN in the research institute where I work. I was considering install it following this doc (datacats way) : http://datawagovau.readthedocs.io/en/latest/ckan.html and use your fork extensions : florianm/ckanext-spatial and florianm/ckanext-scheming

Do you think it's the best option ? And do I have to install also the ckan version of your department (and then adapt it) : florianm/ckan ?

[florianm]

@acheype thanks for your interest!

We're running a CKAN instance following the above setup, however since I've pulled a few updates from CKAN master I must have introduced some performace blocker.

To make things worse, datacats have since ceased to exist, and their SOLR schema (2.3) is far behind CKAN master's required schema.

My setup also still runs as data.wa.gov.au, and Link Digital have productionised it using some neat AWS infrastructure: http://ckan.org/2016/10/13/enterprise-ckan-stack-for-aws-is-now-available-on-github/ I assume they've kept / rewritten / substituted my spatial search but haven't had a chance to look yet.

Meanwhile, I'll sit with my IT ops crew mid November and plan the future of our internal CKAN setup, and if the powers in charge of my schedule are willing, I'll get to rewrite a clean version of -spatial and -scheming and maybe get it to PR level - but can't promise anything ;-)

I estimate that either a vanilla CKAN or Link's architecture might be the quickest way to a georeferenced CKAN.

If you would like to ask the CEO of Link Digital directly, he'll be at this meetup (Thu 1.30pm Canberra time): http://www.meetup.com/Asia-Pacific-CKAN-Meetup/events/234493895/

hope that helps!

[acheype]

Thanks for your instructive answer @florianm! It really helps !

Good to know that datacats is dropped. I don't need to have a cloud architecture so I will directly begin with a vanilla CKAN. Also, it would certainly be easier for me to learn about the structure of a CKAN instance. Do you think your forks ckan/ckanext-spatial and ckan/ckanext-scheming are still compatible with the last version of CKAN ?

And of course, I will keep me informed about a future PR from you. It could be really awesome to finally offer a real georeferenced CKAN "out of the box".

[howff]

Is there any update on this capability? @florianm repos are gradually diverging from ckan master.

[florianm]

Hi @howff - I never got around to finish up the PR to ckanext-spatial and -scheming. Ideally this should become a part of ckanext-spatial but the complication is that it also uses ckanex-scheming. The fact that we're behind master is purely due to limited bandwidth to maintain. Having said that, we're using the above happily in production, but are looking at a CKAN upgrade within the next 6 months. Would you be interested in contributing? I might get some bandwidth for this early 2020.

[howff]

Thanks @florianm I would love to help get this pushed into master. The ckan guys are happy with the fact that it touches both scheming and spatial. Please let me know what I can do right now to get it working and bring it up to date ready for a merge. Happy to take this offline if you want.