ODK Collect can capture points (geopoints), lines (geotrace), and polygons (geoshape).
ODK Central's 0.7 OData submissions export geopoints either as GeoJSON (ODK Central and ruODK default) or WKT.
ODK Central's 0.7 OData submissions export geotrace and geoshape always as GeoJSON.
Since v0.8, GeoJSON and WKT export seems to work across all geo field types.
What does a data analyst want to do with each location type?
How much parsing should ruODK provide?
As a data analyst, I need to:
extract simple lon/lat columns from each geo type - easy for points, harder for trace/shape: centroid lon/lat, or first point lon/lat
export data as spreadsheets to share with minimally tech savvy users - requires lon/lat in separate columns for each geo type
create a map (leaflet) and put geopoints, geotraces, geoshapes on it - requires a format leaflet can map plus working examples in a new vignette "spatial is not special"
Which formats should ruODK output:
sf::sfc_POLYGON via sf::st_as_sfc() (note sf github actions is broken, introducing sf to ruODK will shaft GHA)
st::SpatialPolygons via rgeos::readWKT()
data model considerations: ODK forms can have multiple location columns, R spatial objects have exactly and up to one geometry column
performance consideration when creating spatial types (by attribute or by feature?)
Implementation
Current behaviour
GeoPoint if WKT is split into _{longitude, latitude, altitude}
GeoPoint if GeoJSON is split during rectangling into anonymous fields _{11,12,13} (or other numbers)
Geotrace and geoshape are not handled and remain GeoJSON
Suggested new behaviour (discuss)
Retain the original column to allow parsing into spatially enabled formats (st_point, sfg and friends)
Annotate fields with _{longitude, latitude, altitude, accuracy}
For geotraces and geoshapes, which point should be extracted? Options: first point, centroid. I'd go for first point, as that's actually part of the shape.
An obvious scope boundary is introducing new dependencies. If we just extract point coordinates we don't need sf, st, rgeos, rgdal which can be a pain to install. We could provide examples for going full spatial as a starting point.
Probably out of scope
Do more with the geopoint - turn into sfg (simple feature geometry) in addition to splitting out coordinates (great to have for leaflet maps)
Parse geotrace and geoshape into sfg (keep GeoJSON? always parse?) First cut here.
Challenges
odata_submission_rectangle needs to exempt geo{point, trace, shape} from unnesting
odata_submission_rectangle will blindly (as it has no form introspection ...yet) unnest any list column. This works OK-ish for GeoJSON points as these have a set length. Unnesting geotraces and geoshapes will inevitably end in tears over their variable number of points.
This means that odata_submission_get(parse = TRUE, wkt = FALSE) will fail with geotraces or geoshapes.
Therefore odata_submission_rectangle needs a parameter form_schema = NULL to pass an optional form schema, extract field names of geo{point, trace, shape}s, and exempt those from rectangling. This would preserve GeoJSON and parse GeoJSON into list columns, which then could be further parsed into spatial classes.
Getting too funky with spatial classes will add dependencies
Downside: sf, st, rgdal, rgeos dependencies could be a pain for users to install.
Upside: Adding some nice helpers and working examples (mapping, spatial operations, coordinate extraction) would be immensely useful to less spatially versed R users.
As DBCA still runs a production server on ODK Central v0.6, and the spatial output has changed since then, I will address this issue after migrating our server. ETA mid 2020.
Checklist
Once spatial parsing has changed, make sure to update all the docs and examples.
[x] Add example data containing all geofield types for each option: GeoJSON / WKT, parsed / raw, plus form_schema. Use in tests and examples. Keep example data up to date with package functions.
[x] Add source for geofield example form to inst/extdata
[x] Function examples (odata_submission_get, _rectangle, handle_ru_geo*)
odata_submission_rectangle excludes spatial fields from unnesting if given a form_schema
ruODK handles geopoint, geotrace, geoshape
original values are parsed but retained (GeoJSON > nested list, WKT > text), additionally, point coords (using first point for trace and shape) are extracted into lon, lat, alt, and acc (where given)
location test data included
vignettes are updated (WIP - good enough for now but added comments with ideas for improvement)
Spatial musings
ODK Collect can capture points (geopoints), lines (geotrace), and polygons (geoshape). ODK Central's 0.7 OData submissions export geopoints either as GeoJSON (ODK Central and ruODK default) or WKT. ODK Central's 0.7 OData submissions export geotrace and geoshape always as GeoJSON. Since v0.8, GeoJSON and WKT export seems to work across all geo field types.
What does a data analyst want to do with each location type? How much parsing should ruODK provide? As a data analyst, I need to:
Which formats should ruODK output:
Implementation
Current behaviour
Suggested new behaviour (discuss)
Probably out of scope
Challenges
odata_submission_rectangle
needs to exempt geo{point, trace, shape} from unnestingodata_submission_rectangle
will blindly (as it has no form introspection ...yet) unnest any list column. This works OK-ish for GeoJSON points as these have a set length. Unnesting geotraces and geoshapes will inevitably end in tears over their variable number of points.This means that
odata_submission_get(parse = TRUE, wkt = FALSE)
will fail with geotraces or geoshapes.Therefore
odata_submission_rectangle
needs a parameterform_schema = NULL
to pass an optional form schema, extract field names of geo{point, trace, shape}s, and exempt those from rectangling. This would preserve GeoJSON and parse GeoJSON into list columns, which then could be further parsed into spatial classes.Getting too funky with spatial classes will add dependencies
Downside: sf, st, rgdal, rgeos dependencies could be a pain for users to install.
Upside: Adding some nice helpers and working examples (mapping, spatial operations, coordinate extraction) would be immensely useful to less spatially versed R users.
Testing
Test form: https://sandbox.central.getodk.org/#/projects/14/forms/build_Locations_1589344221/submissions
Added to CONTRIBUTING.md / Test:
As DBCA still runs a production server on ODK Central v0.6, and the spatial output has changed since then, I will address this issue after migrating our server. ETA mid 2020.
Checklist
Once spatial parsing has changed, make sure to update all the docs and examples.