xquery support

[litlfred]

This is a ticket track a discussion on considering XQuery support within ODK. See for an example here on using xquery and xforms: https://en.wikibooks.org/wiki/XQuery/Simple_XForms_Examples

[litlfred]

List of open-source xquery implementations: http://www.martinbroadhurst.com/open-source-xquery-implementations.html I have used BaseX before. It can be embedded w/in a java app or be used a basis for web-services with RestXQ: http://exquery.github.io/exquery/exquery-restxq-specification/restxq-1.0-specification.html http://docs.basex.org/wiki/RESTXQ

[MartijnR]

Fancy! I think it would be great if you could give some examples of the capabilities we'd gain over the current XForms (perhaps extended with full XPath 1.0 support).

[litlfred]

I'm not sure if these are applicable for ODK or not, but there are a few things. If you use one of the built-in data stores that BaseX or exist-DB for example offer, then you don't need to do any mapping of XML data to a relational database (and back). This drastically reduces software development time.

If you're doing any transformations from XML to XML (for example rendering a XForm as HTML, or produce something that will be rendered on Android client etc., or data transformation for other standards like HL7 FHIR in the health domain) then XQuery is a really powerful tool to consider over something like XSLT. The code will produce Will be a lot more manageable & legible and likely significantly shorter.

[lognaturel]

@litlfred I know it can feel a bit artificial but we're trying to separate implementation concerns from the XForms specification since we have some pretty different tools that implement the spec. That means there are two separate changes to consider -- an internal change to using an XQuery implementation (in JavaRosa, Enketo, etc), and a spec change to surface XQuery or some subset to form developers and tools like XLSForm. We could presumably use XQuery internally without making any spec changes.

So I think for this conversation the big question is how would forms and form developers benefit from XQuery support?

[litlfred]

Sorry, I am not sure what the ODK data workflows, what is in scope are and what pain-points problems you are trying to solve, so I am not sure whether or not xquery make sense. Where I have found xquery useful is if there is some required (by the spec) pre- or post-processing of a form submission. As an example, you could use xquery to transform and submit collected data to a third party API in a post-processing step, or you could you use xquery to pre-process an xform before sending it to a rendering engine.

In terms of the problems that you are trying to solve, I can only share my personal experience. I was involved in the development of a healthcare standard: http://www.ihe.net/uploadedFiles/Documents/ITI/IHE_ITI_Suppl_CSD.pdf (see page 40) that used XForms (and incidentally XQuery). I had hoped to quickly connect in the ODK stack for this as part of a reference implementation. Unfortunately, the sort-of "xpath 1.0 and 2.0" meant that I would have essentially had to write a parser to deal with the ODK xforms spec. This was a blocker for going forward with this part of the implementation.

I totally understand the desire to keep implementation details out of the spec (I worked hard in the above standard to make sure that an XQuery based implementation wasn't required). At the same time, I do think it is good to look at the deployment environments you expect to use the standard in. It would be really great if the evolution of the ODK XForms spec allowed people to use already built libraries. For example, in reference to https://github.com/opendatakit/xforms-spec/issues/96, the last time I checked there wasn't an XPath 2.0 implementation available in a standard unix environment outside of a java environment (e.g. Saxon or BaseX) as libxml only support xpath 1.0: http://xmlsoft.org/