Optimize the Form Metadata API when a single provider is used

[obruchez]

Currently, all the filtering, sorting, merging, and pagination logic is implemented in the proxy. All form metadata are loaded in memory and then processed.

If no remote server is specified and only a single local provider is defined, then it might be possible to do the filtering, sorting, and pagination in SQL.

The resource provider returns an empty list of form metadata. Calling that provider first and checking that it indeed returned no form metadata should then make it possible to decide that we can optimize the request using SQL.

[obruchez]

The following metadata are contained in the form_metadata column as XML:

• status/availability
• title (one or multiple languages)
• operations

Can we really filter/sort on those values?

[obruchez]

Think about possible optimizations when multiple providers / remote servers are present ("streaming" of data with early stopping when all necessary results have been retrieved, etc.).

[ebruchez]

If I remember some earlier ideas:

• sorting should be done in SQL
• filtering could be done in SQL or Scala
• pagination could be done in SQL or Scala

But if at least sorting is done in SQL, then it is possible to write a stream merging algorithm that will be as efficient as can be. With a single provider, obviously; but with multiple providers, this could pull just as much information from the providers as needed.

[obruchez]

If a provider only supports version 1 of the Form Metadata API (GET requests), the proxy might also want to fallback to using GET requests and processing the results in memory as is done currently.