Embedded XForms Server for mobile and offline clients

[ebruchez]

Status

The purpose of this RFE is to gather ideas and track progress of an Orbeon Labs investigation project. The purpose of the project is to make the Orbeon Forms forms engine available on clients, including online and offline web browsers and mobile applications.

As of October 2016, there has been some progress on this project, including:

• conversion of our DOM library to Scala
• build modularization
• experimentation with XPath 2 in the browser

We expect to continue progress on this project incrementally.

Subprojects, dependencies, and related projects

• [x] XPath 2 processor/API with JVM and Scala.js support #2127
• [x] XPath: replace reflective calls with native functions #2214
• [x] Don't use dom4j XPath APIs #2164
• [x] Migrate to a minimal version of dom4j #2165
• [x] Complete conversion of XForms engine to Scala #2825
  • did all that's needed, with some leftovers

These are related subprojects

• Requirements for an offline iOS/Android app #1545
• Mobile/offline Form Runner UI

Historical note

With Orbeon Forms 3.7 (back in 2009!), we had an implementation which wasn't bad in concept, but was too limited and based on Google Gears (the precursor to HTML5 local storage among other things). Due to those limitations, Google abandoning Gears and lack of interest from customers at that time, we decided not to maintain it anymore starting Orbeon Forms 3.8. See:

• XForms - Offline Mode Documentation
• XForms - Offline Mode Internals
• How can I create forms which users can fill out offline?

  Rationale

The idea is to bring our forms engine to Web apps and "native" mobile apps. This would serve multiple purposes:

1. supporting offline form management and data entry with a Web app
2. supporting offline form management and data entry from with a native phone/tablet app
3. generally having more of the form processing handled on the client, as client capabilities improve

The general idea is to make the code which currently runs on the server run on the client, whether within a Web browser directly or as a native application on a mobile device.

We need to abstract and modularize the XForms engine to make it as small as possible and as independent as possible from other things including the servlet API.

The user-interface part would remain browser technology-based. We don't want to change this at all at first and we wants to leverage either an actual Web browser or a Web view within a native app.

One components that we need to have is some kind of abstraction for the client/server communication. Right now that communication is handled via Ajax in the JavaScript code and we have the XForms server on the server. What we would like to do here is allowing the option to have something local like at simple queue. That doesn't seem to be extremely hard to do.

Platforms

General considerations

• We plan to compile our code to JavaScript with Scala.js.
  • it is no longer experimental as of 2015
  • it can run in a WKWebView as of iOS 8
• We have a mix of Java and Scala code and we plan to migrate the Java code that is needed to Scala. (#2825)
• We are ruling out the use of GWT, which is a tool chain made obsolete (for us) by Scala.js.

  Web app

For a web application we have to compile our code to JavaScript. For offline support we can use modern browser APIs.

Native iOS app

The current plan is to use JavaScript as well. In the future, and if it matures and gains the appropriate bindings, Scala Native could become an option as well.

HIstorical considerations

We also considered these options, but have opted not to use them:

1. Scala -> scalac -> IKVM -> Xamarin.iOS -> ARM x86 assembly
   • iKVM is stable
   • MonoTouch is stable (?)
   • uses optimizing compiler (I think)
   • drawback: painful toolchain
   • benefit: might be the best performance possible as of 2013
   • opens the door of Microsoft phones/tablets as well
2. Avian VM
   • promising (2013-08-30 update: Scala runs fully)
   • JIT/AOT compilers and GC performance still poor

Abandoned projects, for reference:

1. RoboVM compiles Java bytecode to native code (project/product is dead)
2. ScalaGWT
   • run in WebView
   • no JIT in WebView probably
3. LLVM backend
4. J2ObjC from Google
   • compiles Java source to Objective-C
   • see also Going under the hood of Inbox
   • would not work for us almost half of our server-side code is in Scala

     Native Android application

Android already supports Java bytecode. This means we have two options:

1. Compile our Java/Scala code to Android.
2. Compile our Scala code to JavaScript as in the case of the browser and iOS.

We would need to use a WebView probably and then we would need to interact between the application and the WebView. On Android the JavaScript side of a WebView is able to call a Java object that has been exposed.

Modularization effort

The Orbeon code needs to be modularized and abstracted. In particular:

• client-server connection
• temporary files
• form compilation vs. form execution
• HTTP client for submissions
• etc.

  Subsequent steps

Once a prototype is developed, we needs to think about the general architecture of the mobile application, including lifecycle, how to load forms and so on. Form Runner needs to be included, and local storage at least needs to be used.

[ebruchez]

Steps for a prototype

Strategy

As of 2015-01-13, the plan to use Scala.js. This way a single compilation can be used:

• in standalone web browsers
• in native apps via web views

This also allows us to master Scala.js, which we want to use more for Form Builder (and later Form Runner).

First goals

We are trying to define reasonable chunks of work which lead us somewhere. First goals:

1. run some of our code in the browser
2. run an XForms hello world prototype in the browser

It is ok to hack to get to these goals. The purpose is to get familiar with the tool chain, potential issues, and learn more about what remaining work will be needed.

Known challenges

• Java code/libraries don't compile with Scala.js right now
  • [x] XPath 2 processor (#2127)
  • [x] dom4j (#2165)
  • [x] joda-time/java-time
  • Scala.js port
  • [x] Apache commons
  • [x] commons-codec
  • [x] commons-io
  • [x] lang3 (#2944)
  • [x] our own Java code (which we will migrate) (#2825)
• handling of uploads will be different
• handling of submissions will be different
  • 2016-09-16: Noting RosHTTP which is 100% Scala and works on the JVM, Node.js and browsers. (not maintained)
  • 2020: can use Fetch API directly
• modularize/refactor
  • we only want to pull in a small subset of our code

Things the client must not need

• XSLT 2
  • later dynamic XBL might require a solution, preferably NOT using XSLT
• XPL and processors
• eXist, iText, Flying Saucer
• most other XML processing code (MSV, XML Schema validator)
  • XML Schema validation could be added at a later time
• Java libraries: httpclient, ehcache
• Java logging libraries (need some kind of wrapper)
• at first: no HTML handlers
  • client receives always same HTML "template" from server
  • so no full updates at first

General architecture

• XForms
  • what we really want to run
  • XFormsContainingDocument, model, control tree, control comparator, and related
  • so basically the core XForms engine
  • client takes
  • HTML page produced by server
  • serialized version of static state
    • could be XML or JSON or anything that can rebuild ElementAnalysis trees
    • this way complex XBL logic, XSLT etc. remains on server
  • client initialization
  • all controls are non-relevant
  • initial show does a diff with fully non-relevant tree (null)
  • client operation
  • state handling is changed
    • static state
    • explicitly kept, as it represents the "form definition taken offline"
    • dynamic state
    • app is not multi-user
    • there could be only one such dynamic state at any given time
    • later, we could support switching between form sessions, keeping a number of them
• Form Runner
  • persistence
  • use some kind of local storage/db on the phone

Q&A

• Q: Why wouldn't we compile the form on the client too? A: Form compilation right now requires XML, SAX, and XSLT 2, in particular for some key XBL components. We can't rule this out in the future, but for now it doesn't seem unreasonable to say that forms are compiled on the server, and that the client gets HTML + serialized data structures that it needs.

[ebruchez]

[ebruchez]

+1 from customer

[ebruchez]

+1 from evaluator

[ebruchez]

Immediate concrete next steps:

• [x] create an XForms module in the build (we have progressed a lot on build modularization already) (#3292)
• [x] convert at least P1 classes to Scala (#2825)
• [x] after that: determine next step for further modularizing XForms engine

[ebruchez]

+1 from evaluator, with a preference for a web app using browser storage, as native apps can be more challenging to deploy.

[ebruchez]

+1 from customer

[ebruchez]

+1 from evaluator

[ebruchez]

+1 from user

[siebertm]

We as a customer would also +1 that. Just the form runner, right?

[avernet]

@siebertm Correct: this feature is specifically for forms created with Form Builder and running in Form Runner.

[ebruchez]

+1 from user on StackOverflow

[ebruchez]

This feature will also help us write more responsive UI for Form Builder. We'd like, for example, Form Builder dialogs showing and navigating faster and reducing the number of roundtrips to the server.

[ebruchez]

The proxy portlet preferences could also benefit.

[antoinesamaha]

Hello, Is there any news regarding offline form runner? It is really a requirement from our business people. On the other side how does the Form builder store the form layout. Is it a JSON or XML file? And is the response also an XML JSON file? In which case one would imagine coding a new renderer for mobiles and Flutter can be an option for cross platform compatibility.

[avernet]

@FOC-framework Offline support is still something we'd like to do, but just not actively working at this point in time. Form definitions are stored in an XML format, which essentially looks like HTML + XForms + some Orbeon Forms specific elements and attributes. You can see what it looks like by going, in Form Builder, to Advanced / Edit Source. For services called by the forms, Orbeon Forms supports both XML and JSON.

[ebruchez]

Moving and reworking the steps into this separate comment:

Proposal for next steps

1. Initial XForms engine extraction

Goals:

• must compile with scalac only (no Java code)
• must only use Scala code/libraries
• exclude XSLT, XPL, HTTP client, etc. (everything not core XForms)
• get a mostly extracted XForms engine
• get a list of remaining library dependencies if any
• get a list of further tasks that might come up
• if possible: do this as refactoring which can be merge on master branch

Non-goals:

• remove all Java dependencies
• compile with Scala.js

Steps:

• move things around until they compile ;)
• write interfaces & stubs where needed, hacking is ok

2. First compilation with Scala.js

Goals:

• reasonable subset of XForms engine compiles with Scala.js
  • XPath dependency analysis is explicitly excluded
  • use DumbXPathDependencies
• clarify next steps

Non-goals:

• code doesn't need to run or do anything useful yet
• having a functional XForms engine

Steps:

• if needed, stub out XPath engine, BUT could also hookup Darius XPath

3. Simple client-side Hello World

Goals:

• run Hello World in the browser

Non-goals:

• fully functional XForms engine (no full XPath support)
• process XForms source on the client

Steps:

• client/server infrastructure
  • serialize/deserialize static state
  • generate initial HTML with serialized states
  • hookup local Ajax server
• hookup XPath processor (Darius XPath)
• interface with DOM for the UI

4. Full XForms support

Goals:

• most XForms must work
• complete XPath support
  • XPath 2 processor/API with JVM and Scala.js support #2127
  • Java/Scala bridge for function calls without reflection
  • typed values
  • hookup XPath function library
  • dependencies (PathMap) support
• complete XForms support that makes sense
  • submissions
  • TODO

Steps:

• TODO

5. Initial Form Runner support

Goals:

• some forms built with Form Runner must work
• basic support for taking a Form Runner form offline and sync back online

Steps:

• TODO

6. Full Form Runner support

Goals:

• Requirements for an offline iOS/Android app #1545

7. Option: demo iOS app

Goal:

• run the above in a simple iOS app
• must show basic XForms, interactive, in web view

Steps:

• TODO

[siebertm]

@ebruchez that's a large chunk of work and i know this is rather far out but if you're at the point you've got an MVP or so to test out, ping me! I really want to try this in our real use cases!

[ebruchez]

The initial refactoring requires separating the following:

• [x] form runtime
• [x] form compilation
• [x] submission module
• [x] client/server communication
• [x] TODO

[ebruchez]

• [x] eventually: two separate subprojects, for example xforms-runtime and xforms-compiler
• [x] XFormsContainingDocument → Scala
• [x] static state: separare construction from data structures

[ebruchez]

We will need the handlers offline to support instantiating new forms offline. Right now, this works as follows:

• take the SAXStore with the template
• replay this through the handlers system'
• the output is SAX events
• that's then serialized

So we need a way to do this when offline as well. We probably need to have SAXStore and some of the SAX API for this.

[ebruchez]

Splitting this into modules/projects is not that trivial. We not only have the runtime/compile-time axis, but the "thin client" (what we have now)/full client axis, and the JVM/JS/shared axis.

For example we don't want the "thin client" to pull down the full client. Scala.js prunes the call tree but it's not perfect at doing that, probably, so it's better if we have a clean separation of the "full client" module, which would contain mainly the XForms runtime.

[ebruchez]

How static state construction works:

• XFormsStaticStateImpl.restore / .createFromStaticStateBits / .createFromDocument (tests)
• new PartAnalysisImpl
• topLevelPart.analyze()
  • new RootControl
  • recursive call to build()
  • final hooking up

[ebruchez]

Client-server communication abstraction:

• AjaxClient must be abstracted or call to a common interface
  • currently this depends on JS
• we probably keep the event queue in all cases
• we need a shared event representation ideally (so later, the client-server protocol can use those too)
  • currently AjaxEvent depends on JS
• ClientServer still extracts information from Element

[ebruchez]

So for the client-server, we now have:

• ClientServerChannel
• WireAjaxEvent
• The response is still XML and will have to be updated as well.

For XFormsModelSubmission:

• RegularSubmission uses Connection, which uses PropertiesApacheHttpClient and InternalHttpClient
• neither client is appropriate when offline
• but when running fully in JS, something like a FetchHttpClient would be good, although it would have to be async
  • See "Switch the actions implementation to use async submissions #4077"
• when fully offline, we should have an OfflineHttpClient, configurable to call back JavaScript functions

So probably that we don't need to modify much XFormsModelSubmission. One approach:

• make Connection's choice of HttpClient pluggable
  • on the JVM, use PropertiesApacheHttpClient and InternalHttpClient
  • on JavaScript, use OfflineHttpClient
• remove a few dependencies
  • org.apache.http.client.CookieStore/Cookie
  • etc.

Another approach is to make Submissions pluggable, and on JavaScript:

• remove RegularSubmission, and possibly others
• instead, use a new OfflineSubmission

The second approach might be easier as it's not pulling in Connection.

[siebertm]

My 2 cents on this:

when fully offline, we should have an OfflineHttpClient, configurable to call back JavaScript functions

Why not normal fetch/XHR which are then caught in a ServiceWorker? This generates less special cases.

Depending whether this must support IE, of course. But, even we (working with german hospitals) are phasing out IE support...

[ebruchez]

@siebertm Thanks for the feedback.

Why not normal fetch/XHR which are then caught in a ServiceWorker? This generates less special cases.

The main reason for a JavaScript callback API is embedding in native mobile applications which can then provide their own persistence and service support. It's also very easy to implement on both sides.

For regular online/offline mode in a browser then yes, Fetch support is needed.

Depending whether this must support IE, of course. But, even we (working with german hospitals) are phasing out IE support...

Agreed, and IE support is not a requirement at this point.

[ebruchez]

For the record, we have now made Connection implement ConnectionTrait, with two implementations (JVM/JS). The JS implementation is not done yet.

More work:

• [x] CacheableSubmission depends on XFormsServerSharedInstancesCache
• [x] provide implementation of Connection based on API callbacks and/or Fetch

[ebruchez]

Which parts of the static analysis must not be present in the ElementAnalysis tree?

• XPath analysis
  • this is done lazily (I think) in SimpleElementAnalysis
  • we could do it on the client, I guess, but we shouldn't need to
• calls to ComponentControl.setConcreteBinding, which uses XBLBindingBuilder
  • this is the big one!
  • ShadowChildrenBuilder, which is implemented by ComponentControl, and uses binding.compactShadowTree

So we should probably separate the "builders": ChildrenBuilderTrait, ShadowChildrenBuilder, etc.

[ebruchez]

So now we are refactoring out all the building of the tree. This will take care of XBLBindingBuilder and others.

However we need to solve the question of how to pass in the XPath analysis information. We do this after we have built the tree:

• rootControlAnalysis.analyzeXPath() // Analyze root control XPath first as nested models might ask for its context
• analyzeModelsXPath() // Analyze all models XPath
• analyzeControlsXPath() // Analyze controls XPath

For deserialization, the information should be passed to the constructor. So maybe the constructor of the classes should have the XPath analysis information as vars set to None, but then the vars are updated before serialization. Upon deserialization, the proper information can be passed.

[ebruchez]

Next step is to move towards serialization/deserialization. This includes:

• ElementAnalysis hierarchy
• Scope
• Metadata: must include Mark, NamespaceMappings (but no IdGenerator, XBL index, etc.)

We can use case classes with vars, or semi-automatic, or manual serialization.

[ebruchez]

Remaining tasks before we can rebase on the pe branch:

• [x] fix all tests on offline branch
  • [x] OrbeonClientTest
    • 2021-06-04: started looking
• [x] resolve Saxon HE contributions
• [x] clean existing commit log (for example "xxx" in logs)
• [x] remove printlns
• [x] check freeTransientState calls/frees everything

On master (CE) branch:

• [x] remove xformsOffline
• [x] remove formRunnerOffline

P2:

• [x] FormRunnerSimpleDataMigrationFunctionLibrary
  • 2 functions

[ebruchez]

For 2021.1 this is not a full-fledged feature. Our plan is to enable, possibly optionally, the "Test Offline" feature from Form Builder. This feature allows running a form in test mode using the JavaScript form runtime.

[ebruchez]

• [x] move P2/P3 to follow-up issue
• [ ] doc