Feature request: spatial widget

[florianm]

Ok this is stretching it a bit, but since scheming tackles custom form widgets, and ckanext-spatial provides this really good to have but (for end users) quite challenging spatial field, it would be very useful to integrate both.

A form_snippet spatial.html should allow users to

• paste a valid GeoJSON geometry as per ckanext-spatial, or
• draw a polygon on a leaflet map, writing back the geojson geometry of the convex hull (if several polygons are drawn) or
• upload a geojson file with the bounding polygon (or a bunch of features), then save the convex hull's geometry

A display_snippet spatial.html could render the dataset extent just like ckanext-spatial does as a map, hiding the unhelpful WKT blurb. Edit: done

Also posted to ckanext-spatial.

[wardi]

huge +1 from me. It should be part of a different extension though because it's useful even if you're not using scheming. ckanext-spatial seems like the most appropriate place.

ckanext-fluent and ckanext-repeating are examples of extensions that come with ckanext-scheming support for the field types they add.

[florianm]

Cheesy mockup. https://github.com/florianm/ckanext-scheming/blob/master/ckanext/scheming/templates/scheming/form_snippets/spatial.html

1) I have it - paste geojson

2) I know it - draw polygon (needs leaflet plugin, js, css, and save method)

3) I know someone who's got it - calculate from attached geojson resource (needs python/geos)

[wardi]

if this is built a separate extension the JS, CSS and geos lib requirements aren't a problem. What's a "save method"?

[wardi]

We could have a super basic version of this in scheming that just allows entering a bounding box lng/lat values, then another extension can add all the awesome stuff you've shown

[florianm]

Ok, I've moved the mapping magic to a macro - which plugin would be the best place for my macro? ckanext-spatial? New plugin?

A simpler form:

Next steps:

• enable Leaflet.draw on map so user can draw a poly or rectangle (yay bounding box) (need to find a good place for JS, CSS, and custom script to be read after map is intialised)
• let Leaflet.draw's event "created" update the text input ( -> "save method")
• previes any attached GeoJSON resources on the map so the users can trace their study sites, and also apply their silent knowledge of the actual dataset extent (not necessarily the sum of study sites or spatial data). Removes dependency of geos and post-processing, puts onus and satisfaction of getting dataset extent right (enough) on user.

@wardi have you got any ideas whether that's a bug or a feature: I can't seem to access the global helper h from ckanext-scheming, however ckanext-spatial templates have no problem accessing map_config and other things from h.

[wardi]

@florianm spatial might be the best place if it already has all the requirements.

Hmm. I thought h was inserted into everything. I'm using it here without any problems: https://github.com/open-data/ckanext-scheming/blob/master/ckanext/scheming/templates/scheming/display_snippets/select.html

[florianm]

@wardi you are right, h is available in templates, just not in macros. Also, variables won't resolve to values in macros (or I'm doing something wrong). So if it would be ok to not use a macro in the spatial form snippet, the map widget is pretty doable.

• js module spatial-form (over at ckanext-spatial as suggested) provides drawing on map and map-to-form data binding
• form snippet spatial.html
• needed to update ckanext-spatial's leaflet and leaflet.draw versions (spatial_query.js needs an upgrade now as well)

Using the new spatial display snippet, my dataset extent looks, say, like so:

Updating the dataset shows the new spatial form snippet in action:

• The existing spatial extent is shown on the map
• The user can draw any number of polygons or rectangles, the spatial extent text input will be updated with all drawn features whenever a feature is added, edited, or deleted. Let's add one polygon:

The GeoJSON geometry in the textfield is updated to the new polygon, but the old ones are still visible for reference. Adding another rectangle:

The added rectangle turns up in the textarea input. Now if I could display any attached GeoJSON resources, the dataset maintainer could make the best informed guess about the dataset's real extent, and wouldn't need to fiddle with GeoJSON.

@wardi Is that a sane architecture at all? Can we get away without using macros in the spatial form snippet?

[wardi]

That looks really good!

There's no special reason to use macros. That's just the way the current form reuses bits of jinja2 templates, but it's unnecessary when we have one snippet per form field.

[florianm]

Cross post! Here's a screencast of the spatial widget in action. https://vimeo.com/116324887

[wardi]

here I said this should be separate from ckanext-scheming, but this looks so nice I'm changing my mind. It would be great to have a good standard spatial input and view like this in scheming itself, since users are still free to extend or replace it in their own extensions.

If most of this work applies to ckanext-spatial users that might not be interested in scheming, though, then it's probably better not to duplicate it.

[florianm]

I agree, I feel this is more a ckanext-scheming feature than a ckanext-spatial one, as it provides a custom widget for a custom field. However, the widget depends on features which are more at home in ckanext-spatial. If that's ok, I'll need to ask you and the ckanext-spatials for advice on the mailing list.

[florianm]

Hi all,

could I ask you all, especially the ckanext-spatial and -scheming crew for advice.

My CKAN users love georeferencing their datasets, but, being environmental scientists with sometimes limited GIS skills, some struggle with pasting a correct GeoJSON geometry into the "spatial" extra field. So, with guidance from Ian I've prototyped a spatial input widget which you can see in action at [0] and take for a spin at [1] with un/pw "ckan".

The widget still shows the text input for a GeoJSON geometry (enlarged into a textarea), but also adds a map one can draw the dataset's extent on, and a select box for often needed, well-defined areas.

This widget depends on a few components which may or may not belong into ckanext-spatial or -scheming. ckanext-spatial PR at [4]

• A module spatial_form (based on spatial_query)
• Updated Leaflet.draw, which requires an updated LeafletJS (and breaks spatial_query)
• Fixing spatial_query to work again with backwards-incompatible Leaflet.draw

ckanext-scheming feature request at [5]

• A new form snippet spatial.html [3] which depends on spatial_form to render the map, but also renders a select for pre-defined areas, and contains the JS to data-bind the inputs together. Also I've included display snippets, which render the dataset's extent inline (see video), which replaces the dataset's extent as facet in the sidebar.

The whole setup works well enough on my end, but before I can send a clean(er than [4]) PR I'll need your advice on a few points:

Should all components of the spatial widget live in ckanext-scheming, or which components would be appropriate as PR in ckanext-spatial?

What is the official procedure for upgrading JS dependencies in extensions? Is there a more graceful way than overwriting files in /vendor/ ?

As for data-binding the three input widgets (map, select, textarea), should we use heavier artillery like react.js or will we get away with a few manual JS hooks? Note in my prototype I've only bound map->text and select->text, but not select (in scheming) to map (in spatial).

As for the select box, that really should also live in the spatial_form JS module, right? I kept it in ckanext-scheming's form snippet for now, as it's a ckanext-scheming select widget with data coming out of my custom data schema definition. Of course that's clunky and hard to maintain, so I'd rather have a custom vocabulary with an API (for maintenance) and an autocomplete service (for the input), much like tags, but we'll need three values (id, label, GeoJSON geometry/string) instead of two, like tags (id and label). Would it be feasible to have an autocompleting custom vocabulary for spatial entities, and how would I implement this without interfering with models?

Cheers, Florian

[0] Spatial widget in action (apologies for potato quality) http://vimeo.com/florianwmayer/ckan-georeferencing [1] Demo instance http://data-demo.dpaw.wa.gov.au/ [2] spatial_form JS module https://github.com/florianm/ckanext-spatial/blob/91-spatial-form/ckanext/spatial/public/js/spatial_form.js [3] spatial form snippet https://github.com/florianm/ckanext-scheming/blob/master/ckanext/scheming/templates/scheming/form_snippets/spatial.html

[4] PR ckanext-spatial https://github.com/ckan/ckanext-spatial/pull/93/ [5] Feature request at ckanext-scheming https://github.com/open-data/ckanext-scheming/issues/11

On Thu, Jan 15, 2015 at 2:30 AM, Ian Ward notifications@github.com wrote:

here https://github.com/open-data/ckanext-scheming/issues/11#issuecomment-66448057 I said this should be separate from ckanext-scheming, but this looks so nice I'm changing my mind. It would be great to have a good standard spatial input and view like this in scheming itself, since users are still free to extend or replace it in their own extensions.

If most of this work applies to ckanext-spatial users that might not be interested in scheming, though, then it's probably better not to duplicate it.

— Reply to this email directly or view it on GitHub https://github.com/open-data/ckanext-scheming/issues/11#issuecomment-69965596 .

[MandanaMoshref]

@florianm Is it possible in your approach to adding marker too? As far as I know, the FeatureCollection and GeometryCollection is not supported through ckanext-spatial (https://github.com/ckan/ckanext-spatial/issues/152#issuecomment-252630806). In your "spatial_form.js" you aggregate all geometries into multipolygon. I would like to have both multipoint and multigeometry but in a way that avoid a mix of both (geometry collection type). So for example, if someone picks a marker automatically leaflet deactivates the BBOX and polygon or vise versa and aggregate corresponding features (multipolygon or multipoints). Can you advise me on how this can be done in spatial_form.js? Thanks Mani

[florianm]

@MandanaMoshref thanks for the follow-up! This PR is pretty old and I ran out of bandwidth to bring it to a clean state. We've been using my fork in production since then but are looking to update our setup.

To explain my design choice of using Multipolugons, these simply approximate our datasets best. The raw data are mostly observations with a location (e.g. CSV with lat/lon coordinates and map resource preview), the sites are mostly polygons (e.g. GeoJSON again with a map resource preview, also they show up on my widget), and the dataset extents featured in this PR are the convex hull (polygon) possibly with gaps (multipolygon) around the sites our data come from. Therefore it never made sense to georeference an individual dataset by a point alone. I acknowledge that your use case might be different!

To answer your question, if you could rewrite this PR to use a FeatureCollection instead of a Multipolygon, you may be able to also represent your datasets with a point. You might even get away with leaving all controls enabled.