Interplay with mori?

[comnik]

Hi,

I have integrated datascript from a vanilla JS application. Since we use the mori library for data structures and manipulation anyway, I was wondering about directly accessing cljs results from datascript via mori. This would be very nice to utilize the cljs equality semantics (e.g. for speeding up the React diffing).

Reading the datascript source, I figured it would suffice to disable pull-result->js and the clj->js calls in js.cljs.

Am I making a conceptual error here? And if not, could there even be an easy way to provide this "optimization" optionally through the datascript API (to avoid forking)?

In any case, thank you for the amazing work on datascript! Cheers, Niko

[comnik]

Update: Disabling the aforementioned function calls in js.cljs seems to cause no troubles. Something else does: Minification is causing the JS objects representing cljs values in mori to be different from those in datascript, even though they contain the exact same information.

[typeetfunc]

@tonsky Are there any reasons to use conversion clj->js? Major part of javascript community uses immutable data-structure(for speeding up the React rendering).

[comnik]

For completenesses sake:

I did a quick experiment, exposing all of mori directly through datascript (to have google closure munge them consistently). As mentioned above, I disabled the clj->js conversions, but kept the keyword -> string mapping. This produced the results I hoped for, allowing me to implement the shouldComponentUpdate optimization as usual.

So it works, but I can't think of a way to make this a nice, optional extension on top of datascript.

[tonsky]

Interesting idea! I can totally see the benefits of using mori together with datascript. You’re right about building it together though — google closure compiler needs to share information about which minifications it does. I can probably get a DS built as a module compatible with specific mori build, but with every change in mori we’ll have to re-build DS to match new minifications.

I think more productive way to solve this is to use DS and mori in their source code form, compile them together, then use.

I think I can add another namespace that will expose API to JS without intermediate conversions. That shouldn’t be a problem

[comnik]

Using mori and DS from source seems like the approach with the least friction. Although that would require using Closure for my own code as well, right?

A direct-access namespace would be helpful in any case, because exposing mori through DS is basically just copying files, changing namespaces and changing the mori-export macro to create Closure symbols like datascript.js.name instead of mori.name.

[tonsky]

Although that would require using Closure for my own code as well, right?

Not necessarily, no. DS and mori can still mark API fns with ^:export so their names won’t be minified with Google Closure compiler.

[tonsky]

Here’s what I did: I marked all fns in datascript.core with ^:export, meaning their names would not be touched by google closure minifier. What you need now is just to run a compiler and point it to both datascript and mori sources at the same time, and use datascript.core functions instead of datascript.js. Let me know if you have any issues with that

[comnik]

Thank you. It might take some time until I get to it, but I'll try it out and let you know.

[kristianmandrup]

Another option would be to wrap the query result on the JS side with an immutable data structure such as ancient-oak

[typeetfunc]

@comnik https://github.com/typeetfunc/datascript-mori. See examples - usage of only CLJS API and usage of combination JS API and CLJS API Also, I have wrote babel-plugin for precompiling EDN string to mori structures(for syntax check and minimize runtime overhead)

[kristianmandrup]

Pardon me for asking, but what is the use case for transforming a Datalog query into a Mori structure? I thought it was mainly useful for wrapping a query result?

[comnik]

@typeetfunc Thank you, I will check that out as well, unfortunately occupied with other projects atm.

@kristianmandrup: Mori, in contrast to other libraries for JS, is a wrapper around ClojureScript's (CLJS) immutable datastructures. Likewise, datascript is built in CLJS, with the API additionally being exposed to JS. In the process, datascript has to convert it's internal, CLJS results into JS objects. In my use-case, I'm tying together datascript and mori in a JS application. That means a datascript result will first be converted to a JS object and then back to an immutable structure (via mori). This is of-course wasteful.

Additionally, React in combination with immutable data, allows for a simple but highly effective optimization, because it allows to compare two state-trees by reference, not by walking through both trees. Unfortunately, I can't make use of this optimization for data returned from datascript (which is all the data in my case), since I'm always creating a new mori object during the conversion, even though the underlying datascript structure might not have changed.

By using in both in combination, as discussed in this thread, I can just cut out the cljs->js->cljs conversion entirely.

[typeetfunc]

what is the use case for transforming a Datalog query into a Mori structure?

@kristianmandrup I add some motivation in Readme

@comnik Great explanation! If you have questions about datascript-mori - let me know.

[kristianmandrup]

"That means a datascript result will first be converted to a JS object and then back to an immutable structure (via mori). This is of-course wasteful."

This was exactly what I was after/questioning as well. So datascript-mori is actually an efficient "bridge" which directly converts to mori without the intermediate to Javascript conversion?

Yes, I'm aware of the benefits of immutable datastructures for application state comparisons for efficient diffs and re-renders. Why I'm interested in this datascript/mori approach! :)

[kristianmandrup]

Thanks @typeetfunc for describing the motivation behind the babel plugin. Could you also add similar motivation, examples etc. for datascript-mori :)

[tonsky]

As of DataScript 0.16.0 I’ve removed externs from datascript.core as they were disabling dead code elimination (see #191). @typeetfunc if you want to continue use datascript.core directly from JS, just add externs fine when building datascript-mori