Explore replacing custom XPath engine with off-the-shelf library

[lognaturel]

Software versions

JavaRosa v2016-04-21

Problem description

opendatakit/xforms-spec#96 and opendatakit/xforms-spec#95 have been exploring including a more standard and complete XPath API in the ODK XForms spec (XQuery is a superset of XPath). There are some practical considerations to take into account -- for example, browsers support XPath 1.0 natively so sticking to XPath 1.0 makes it much easier to have browser-based implementations like Enketo.

JavaRosa currently implements its own XPath engine with lovely comments such as /* if you try to edit this code, you will likely break it */. There are several XPath engine implementations with Saxon appearing to be the most actively developed.

It would be good to understand

• why an off-the-shelf library wasn't used in the first place
• how invasive a change to a standard library would be
• whether it's possible to do a replacement in a way that won't break existing forms

[yanokwa]

@ctsims @mrgriscom Bit of an archaeological question for you. Any idea why an off-the-shelf XPath library wasn't used in JR?

[ctsims]

Hey Yaw!

sorry I haven't been keeping up with these, we've been unbelievably bogged down over here.

JavaRosa was massively constrained by

• The amount of available Heap Memory (roughly 900->1800 kb)
• The size of the Jar file (had to be under 1 mb to run as a Nokia app originally, later it was bumped up to 2mb by Nokia)
• The number of Class Files that could fit in permgen. Once you'd used a class the .class file object gets put in the permgen memory space (shared with the heap) and never freed out, even after GC, so your available heap would drop every time the app used a class. Particularly brutal for libraries which were big on modular or abstracted concepts and had hundreds of classes, since each class had an additional hit on our footprint.

Drew originally implemented XPath using a separate library, then by using a compiled grammar, and both were significantly too large for us to be able to use them.

-Clayton

On Fri, Mar 17, 2017 at 8:27 PM, Yaw Anokwa notifications@github.com wrote:

@ctsims https://github.com/ctsims @mrgriscom https://github.com/mrgriscom Bit of an archaeological question for you. Any idea why an off-the-shelf XPath library wasn't used in JR?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/opendatakit/javarosa/issues/17#issuecomment-287377277, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJdug7tYq9i6GJBWFwnhIpoOda1O4g7ks5rmp90gaJpZM4MgqiO .

[dcbriccetti]

“Saxon-HE (home edition) ... provides implementations of ... XPath (2.0, 3.0, and 3.1)....”

Does the XPath library we use need to support XPath 1.0?