replikativ / datahike

A fast, immutable, distributed & compositional Datalog engine for everyone.
https://datahike.io
Eclipse Public License 1.0
1.62k stars 95 forks source link

create general datahike SQL jdbc backend #161

Closed kordano closed 3 years ago

kordano commented 4 years ago

Instead of using separate backends for sql backends Datahike should have one sql backend which maps directly to the jdbc drivers.

whilo commented 4 years ago

A version of this by @alekcz is available here: https://github.com/alekcz/konserve-jdbc

TimoKramer commented 4 years ago

current jdbc spec https://download.oracle.com/otn-pub/jcp/jdbc-4_3-mrel3-eval-spec/jdbc4.3-fr-spec.pdf?AuthParam=1591192807_8aebdb4779b50c0bbed0cf7f725d8964

TimoKramer commented 4 years ago

should we close this issue? It might take some time until datahike-jdbc is ready to replace datahike-postgres

alekcz commented 4 years ago

@TimoKramer I can get it there over the weekend. Do you have an idea of what's still missing for it to replace datahike-postgres?

TimoKramer commented 4 years ago

Hi @alekcz and thanks! I think the main problem is that datahike-jdbc brings the konserve 0.6.0 with it and when you are using datahike it loads konserve 0.5.1 and then datahike-jdbc does not work. so this is basically a question of when we can use datahike with konserve 0.6.0 I guess. Or making konserve-jdbc work with 0.5.1?!

whilo commented 4 years ago

We need to make datahike work with konserve 0.6.0. We should probably do this after we have released the performance improvements to not block them.

whilo commented 4 years ago

It should actually already work, because we have not changed the konserve protocols to the outside, so it might be just a dependency issue (?).

TimoKramer commented 4 years ago

I think it might work but the question is if you want datahike to use the konserve snapshot?!

whilo commented 4 years ago

No, you cannot use SNAPSHOT dependencies in releases on clojars in fact. But we can do a SNAPSHOT release of Datahike itself with the SNAPSHOT dependency and then release them together once everything fits. I think this is what @kordano asked for.

TimoKramer commented 4 years ago

waiting for https://github.com/replikativ/konserve/issues/31

whilo commented 4 years ago

@TimoKramer You mean you wait for the konserve version, not the garbage collector, right?

TimoKramer commented 4 years ago

@whilo exactly. I thought this is more or less the same.

whilo commented 4 years ago

@kordano and I agreed to use the current konserve snapshot and test all the backends (this JDBC backend, the LevelDB and filestore backend) and if it works, we can release the new konserve version.

whilo commented 4 years ago

So feel free to test the new JDBC code from Datahike and let me know how it goes.

TimoKramer commented 4 years ago

https://github.com/replikativ/konserve-leveldb/issues/4 blocks the usage of konserve 0.6.0-SNAPSHOT

TimoKramer commented 4 years ago

After the PR https://github.com/replikativ/datahike-jdbc/pull/12 is merged. I will change the documentation of using SQL backends in the docs and mark datahike-postgres as superseded.