search

tonsky / datascript

Immutable database and Datalog query engine for Clojure, ClojureScript and JS
Eclipse Public License 1.0
5.46k stars 304 forks source link

Use Datalog implementation with external data and indexes? #22

Closed dahjelle closed 10 years ago

dahjelle commented 10 years ago

At a glance, it looks like it may be possible to separate out the query.cljs portion of DataScript for use with other storage mechanisms, like IndexedDB, if one makes sure to have the proper EAVT, AVET, and AEVT indexes available. Does that seem reasonable? Or does your implementation depend on more than just having those indexes available?

I'm a noob to all of ClojureScript and Datalog, not to mention DataScript, so what I'm asking may not even make sense—feel free to tell me if so. Thanks!

tonsky commented 10 years ago

Indexes are required for fast query lookups and range searches. Slow access (full-scan) is already implemented for collections. IndexedDB would be a good data storage for DataScript, if only it had synchronous api. With callbacks, it seems very hard to use DataScript the ways I imagined it would be used.

boxxxie commented 10 years ago

Use core.async? How hard would it be to add an async API? Maybe a use-channels binding?

tonsky commented 10 years ago

It will ruin the whole “db as a value” idea. You cannot do async callbacks during React render for example. Core.async callbacks will not allow for nested functions. And so on. Barriers where there shouldn’t be any. It will force you to split model and view model, extracting latter from the rendering code. This is a wrong kind of decoupling, because rendering code has high cohesion, it’s a mistake to break these apart because limitations of technology say so.

Also, it’s not clear what benefits of storing data in IndexedDB are?

dahjelle commented 10 years ago

I'd agree with @tonsky that the "db as a value" idea would be sad to lose—I've spent quite a bit of time connecting an async data source like IndexedDB to work with React, and it basically involves duplicating all of the data so that React sees it.

That said, it would be really nice to have a Datalog implementation with the option to hook in external, async indexes (like from IndexedDB or, in my case, PouchDB). I'm giving that a shot—though, since I'm quite the n00b with ClojureScript and Datalog versus JS, I'm just throwing in callbacks. I'll report back with a link to the fork if I get anywhere useful. ;-)

(P.S. If anyone has good core.async examples, I'd love to see them…especially on how to, say, implement in core.async a reduce function whose reducer function is async. As I said, I'm a n00b. ;-) )

dahjelle commented 10 years ago

Well, as I said, I'm quite the n00b with ClojureScript, but here's my fork of Datascript that uses external indexes. Feel free to critique or contribute!