zadean / xqerl

Erlang XQuery 3.1 Processor
Apache License 2.0
64 stars 8 forks source link

Add DB module to allow updating functionality not addressed in the specification #22

Open zadean opened 5 years ago

zadean commented 5 years ago

"a new 'db' module will be introduced that allows for insertion, deletion, and retrieval of any 'item' in the database. This will address all item types, including functions.

What this means: atomic values, XML, JSON, maps, arrays, and functions will all be able to be saved in databases and retrieved later for use in other queries. All atomicity promises will be kept just as with the Update Facility."

grantmacken commented 5 years ago

This is really exciting :+1:

I assume with this module enabled retrieval of resource items in db via fn:collection() function will return a sequence of any stored items, not just XML.

Likewise fn:uri-collection() should list items available, but how will be able to retrieve, the 'type' of the db stored resource items?

I know fn:doc() will be empty is XML document is not available, fn:unparsed-text-available() for 'text' resources, but what else?

zadean commented 5 years ago

Right, fn:collection() will return a sequence of item()* as it already does. fn:uri-collection() returns all URIs in the collection regardless of type. fn:doc-available#1 and fn:unparsed-text-available#1 will return if their respective types are available.

The missing functionality that I'd like to add is to be able to insert/update/delete text and non-XML in the DB. A function similar to fn:doc for any item will also be needed. fn:json-doc can be used for unparsed-text known to be JSON, but isn't ideal for anything else.

Also, a helper function to get the type of the resource could be added. Or maybe a new function like fn:collection() but with parameters for what item types to return.

WillemvdW commented 5 years ago

I discovered xqerl by chance today. Given that it is written in erlang, it is anticipated that xqerl could access data from couchdb that is also written in erlang?

zadean commented 5 years ago

@WillemvdW

it is anticipated that xqerl could access data from couchdb that is also written in erlang?

I was just playing with this idea a few days ago.

Accessing data that is saved in CouchDB is already possible over the REST interface. So writing XQuery that calls the REST endpoints with the HTTP Client. But that can be done by anything.

Another possibility is using xqerl as a transformation layer in the couchdb Query Server. This is actually pretty cool. It means using the Native Erlang Query Server in CouchDB to call XQuery compiled in xqerl directly over RPC. The reason it needs to be RPC is that xqerl and couchdb use different versions of Erlang and won't be running in the same Erlang node.

This allows for complex transformations/aggregations to be written in XQuery and the results given back to couchdb to serve from views or List Functions.

mgaerber commented 4 years ago

Since it came up in this thread: xqerl and couchdb via RPC https://gist.github.com/zadean/1f75901f09a2aa2ba482a2ced00b2a63