zio / zio-quill

Compile-time Language Integrated Queries for Scala
https://zio.dev/zio-quill
Apache License 2.0
2.15k stars 346 forks source link

Integrate a Streaming API into JDBC and other Modules #1228

Closed deusaquilus closed 5 years ago

deusaquilus commented 5 years ago

In order to call it a true one-stop-shop database interaction framework, Quill needs to implement a reactive/streaming API for all it's submodules. Originally in #204, this was concieved as a Monix-based api but then in #985 it was concluded that monix must be moved out of primary Quill modules. I propose that we introduce a monix-based streaming context module called quill-streaming that is implemented by quill-jdbc-streaming, quill-cassandra-streaming etc...

Alternatively, if we implement our own org.reactivestreams.Publisher we get integrations into Monix and many other systems for free.

At this point, I propose that we move streaming out of quill-cassandra into quill-cassandra-streaming and create:

Some other potential candidates are:

Also, it should be noted that @oleg-py has created a library called quill-monix which incorporates monix into quill-jdbc. This seems like an ideal place on top of which to integrate monix observeables!

cemelo commented 5 years ago

I have a quill-monix implementation as well, slightly different from oleg's but pretty much the same functionality. I also have a module integrating refined with quill. Please let me know if I can help with this regard.

Also, since we're talking about functional streams, it would be nice to have errors in monads, instead of throwing exceptions.

fwbrasil commented 5 years ago

@cemelo that sounds promising! is it on github?

deusaquilus commented 5 years ago

@cemelo Yeah, it would be great to have a look. Would you like to create a PR? I'm going to be looking into this next week hopefully.

cemelo commented 5 years ago

They're both still in our codebase, but I can move them into a PR so you guys can take a look. I'll try get that done by the end of the week.

cemelo commented 5 years ago

@deusaquilus @fwbrasil I added a PR (#1261) that contains some code I pulled from our codebase, still rough around the edges but functional.

I also created a repository (quill-contrib) with additional modules that I don't think would be a good fit for the core repo, but might be a basis for an extra kind of repo. Of course, whether that's the case is up to you. 😊 Fyi, there's already a enumeratum-quill library, but that treats enums as strings on the database, while mine uses actual enums (at least with PostgreSQL).

Please, take a look and let me know what needs to be improved and I'll try to work on it.