[Highly Experimental] RSocket Server implementation

[alex-dukhno]

Currently, we use smol crate to handle networking and implementing traits from protocol.

As experimental exercise we can try to implement that part using RSocket Rust implementation

[kuronyago]

would like to take part in the project in this very direction please describe how you see the implementation of the connection on the RSocket, how the structure of the project will be changed according to yours.

[alex-dukhno]

Hi @kuronyago,

RSocket can be applied either internode communication or client - server. It is too early to talk about internode communication as my main focus right now on query execution and transactions. I don't have much to say on this topic except that I am considering to use https://github.com/tikv/raft-rs and RSocket as a network protocol.

That lefts us with client - server communication. Here be dragons... I see two ways. One is to create separate database driver over RSocket, which is huge undertaken. Second is to reuse PG wire protocol. First, I'd start with trying to run some code with https://r2dbc.io driver, that supports PostgreSQL. I don't follow the project closely, however, I assume it should support any database that mimics Postgres over the wire. You can try to adapt compatibility tests to see how database behaves. Saying that, I am not sure that R2DBC use RSocket Java implementation under the hood.

It is hard to comment on full support of RSocket in client - server communication. As I know RSocket can be run over the following network protocols: TCP, Aeron (which uses UDP), HTTP/2 and possibly QUIC (I saw PR to adapt spec for it). Thus, I assume RSocket should somehow be able to run over Postgres protocol ... or it could be negotiated during connection setup and switched to RSocket over TCP (with encryption support as PG protocol natively supports SSL/TLS), but this brings us back to a database driver over RSocket...

Anyway, I am interested in any ideas and suggestions and I'll be glad to listen to them.

P.S.: Bear in mind, that I came to high level design of query execution after 7 months of part-time development of the project 😄 . However, if you are motivated and ready to provide ideas and/or code on how to introduce RSocket into the project, I think it could be done faster 😄

[kuronyago]

I am more attracted to the first option, but did I get it right, do you want to implement R2DBC on Rust for PostgreSQL first?

[alex-dukhno]

do you want to implement R2DBC on Rust for PostgreSQL first?

no, I don't want.

I am more attracted to the first option

Just to clarify by "first option" you mean my:

One is to create separate database driver over RSocket, which is huge undertaken.

Actually, I think it is possible to "reuse" R2DBC ... This is how would I proceed with prototyping client - server communication using RSocket:

• I would setup simple project that uses R2DBC and does sanity check like
  • creates schema
  • creates table
  • make some data insertions
  • selects all data
• Investigated how R2DBC implements its part of communication with PostgreSQL backend and how simple to integrate Java RSocket Implementation into it.
• Then I'd start investigation what could/should be done on database side.

One thing is that I don't know why rsocket-rust use tokio runtime and how much it depends on it but I'd rather not to bring any async runtime into database, because I'd like to have other options on asynchronous client handling for the moment.

[kuronyago]

I'm confused.

if the goal is to use R2DBC, then a crate is needed for the server and the client that will implement this specification with the RSocket Rust implementation.

i.e., in the end will be: 1) a server that can communicate with clients via R2DBC - in progress 2) R2DBC library (crate) implemented via RSocket for client-server communication - as far as I know, there is no R2DBC implementation on Rust yet 3) a client who knows how to transmit commands and requests via R2DBC - in progress?

in fact, neither the client nor the server will know that they are receiving messages via the RSocket channel, as this will be hidden behind the R2DBC crate implementation

I'm sorry if there's something I'm not catching.

[alex-dukhno]

I'm confused.

Ok, I see your confusion but I don't know what exactly the reason of it, so I will try to be clear as much as I can.

First, R2DBC:

1. R2DBC is alternative driver to JDBC to work with databases. JDBC has synchronous interface. R2DBC has asynchronous.
2. Because R2DBC should be used with different databases (PostgreSQL, MySQL, you name it) it has TCK (Technology Compatibility Kit) to ensure that implementation correctly follow specification. If you want to connect and work with Postgres you use r2dbc-postgresql, mysql - r2dbc-mysql
3. r2dbc-postgresql doesn't care to which database it talks to ... PostgreSQL (which is written in C), CockroachDB (which is written in Go). Unless the database "understand" PG Wire Protocol - this means that database can recognise messages that what are sent through the network and can "follow" message pattern, knows what data and how it should be represented by messages so that R2DBC driver could "understand" and interpret results.

Second, PostgreSQL: It doesn't care what sort of clients/drivers is talking to it. R2DBC, JDBC (which written in Java), psycopg (which is written in Ruby) and many others - unless they follow message format and message pattern that specified in PG Wire Protocol.

The same principle applies to applications that use RSocket - because it is a protocol. If we put Java client that talks RSocket protocol and Rust server that talks RSocket - it is going to work.

We don't need new Rust crate for R2DBC. We need integrate R2DBC with Java RSocket implementation to have asynchronous driver that support RSocket [1]. And implement server side (or integrate rsocket-rust) that this project could talk to a driver [1].