Rewrite based on official Java driver?

[gmethvin]

An official Java driver is currently in the works for rethinkdb: https://github.com/rethinkdb/rethinkdb/issues/3930

I think it may be worth considering writing this as a nice DSL layer on top of the Java driver. This way rethink-scala can focus on making a type-safe high-level API, and the low level stuff can be handled by the official Java driver. This also means that even if rethink-scala lags behind the Java driver in terms of ReQL support, it's always possible for Scala users to fall back to the Java types to get things working.

Not sure how popular rethink-scala is, but I'm sure there are more potential Java users than Scala users so having a common codebase will help get bugs fixed faster.

[He-Pin]

+1 for this.

[dustbort]

:+1:

[a1russell]

I'm not sure how much the codebase currently relies on the JVM, but one downside I should point out is the more we rely on Java, the more difficult it would be to get the driver to work with both Scala-on-the-JVM and Scala.js.

And I think this driver would be great to be able to use this driver with Scala.js.

[gmethvin]

@a1russell I think it would be best to separate the Scala DSL for constructing ReQL queries from the backend used to execute them. So it should be possible to translate that query constructed with the DSL into something executable by the Java driver, node.js driver, etc.

[kclay]

@gmethvin

Do you mean making it so that generating the json and sending the actual query ( making connection to db) to be pluggable?

If that's the case it's already possible. There is a "Version" ( maybe I should rename it protocol) and a "Backend" allow you to switch connection provider (right now netty but could be switch to official driver).

The version/protocol take the DSL and converts it to its correct representation, used to have a protobuf but now just json. And the backend does all the IO , parsing to classes ( I maybe should make this part pluggable so other backend can use it).

As for Scala.js @a1russell, I would like to know a bit more on what limitations should I be aware of.. The driver uses a lot of implicits conversions, is that something that Scala.js supports. In some cases I have provided a way to do it without, in other cases it was required at the time to use conversions to generate dynamic classes for a few special cases.

[a1russell]

@gmethvin As I said, I'm not really familiar with the current architecture, so I'm not sure how easy it would be to transition into anything we are talking about. Let me try to elaborate/brainstorm on an approach to achieve what you propose. Hopefully @kclay will find the brainstorming useful.

I could see there being the developer-facing Scala API which operates only on a layer of traits. There would be two implementations of these traits: one which calls the Java driver, and one which calls a second layer of traits that are simply wrappers around the JavaScript driver.

There would also have to be two builds. One of them would create an artifact where the Java driver is used, and the other would create an artifact where the JavaScript driver is used.

@kclay, to answer your question, yes, Scala.js supports implicit conversions. Really the Scala.js documentation is pretty decent, and you may especially want to check out the differences between Scala.js and Scala on the JVM. The most difficult parts of using Scala.js are getting the build file right and either finding a Scala.js compatible library to do what you want or wrapping a native JavaScript one. Supporting Scala.js mostly just means not using JVM libraries, as the list of differences is pretty short, in my opinion.

[gmethvin]

@a1russell I agree that it should be designed in such a way that it's easy to write a scala.js backend, but I don't think scala.js support should be a priority of this project. I'd assume it would only work on node.js since rethinkdb is not currently designed to work client side.

@kclay It would be nice to make the backend completely pluggable, and eventually separate to different libraries.

[a1russell]

@gmethvin Good point about rethinkdb not being designed to work client side. Was just trying to think things through!

[He-Pin]

@gmethvin there is a official js api,but I haven't tried that.but still I think that rethinkdb with java/scala driver will means a lot when it comes with Play/Reactive application.

The js driver maybe only works on the server side,so I think we could make it two phase ,first scala-jvm,then scala.js,just as what you just said.

[kclay]

@gmethvin , I'll work on breaking up the backed a bit more, most of it is already interchangeable . The only part that isn't a clear cut is how Sequences/Streams are built. I do have a working tests with scalaz-streams but its a bit hackish at the moment.

The other part that I need to break apart is the de-seralization of the responses, this would allow your play! issue to be resolved.

My plan for this is to have one library for collections/streams, which can be switched for scalaz-streams, RXJava/Scala and Akka. Then one for the actual connection backend, so default (netty), official and a way to do anything else so making your own connection pools etc. And then another one for serialization, default (jackson), play and maybe a argonaut and json4s one.

Havn't had time to work on this lib in some time but should be able to start back up soon.

@a1russell thanks, I'll do some reading on the scala.js document

[gmethvin]

Havn't had time to work on this lib in some time but should be able to start back up soon.

All the more reason to delegate some of the work to an official library with more people using it. That way at least the mission-critical parts are officially supported and are likely to have bugs fixed quickly.

[a1russell]

Official Java driver is ready now. :smiley:

https://www.rethinkdb.com/docs/install-drivers/ http://rethinkdb.com/blog/official-java-driver/ https://github.com/rethinkdb/rethinkdb/tree/next/drivers/java

[anand-singh]

Congratulations @a1russell

[a1russell]

@anand-singh I don't deserve any credit for anything other than being a messenger. ;-)