search

rsocket / rsocket-java

Java implementation of RSocket
http://rsocket.io
Apache License 2.0
2.35k stars 354 forks source link

Rsocket - Reactive Cassandra less throughput compared to blocking http #758

Open pckeyan opened 4 years ago

pckeyan commented 4 years ago

My use case revolves around retrieving the data from Cassandra using RSocket-Java in a non-blocking way. I use reactive cassandra template built on top of reactor-core. Throughput I see is very low in the range of 300 RPS compared to blocking http calls(800 RPS). If I remove Cassandra and use stub, I am getting close to 5000 rps. Can you please let me know whether RSocket-Java is not a perfect fit for above use cases? Sample non-blocking code is written similar to “https://github.com/OlegDokuka/rsocket-issue-717/tree/master/src/main/java/com/rsocket/examples 1” and Cassandra Connections are configured as mentioned here - “https://dzone.com/articles/reactive-streams-with-spring-data-cassandra”. Please help and advise.

OlegDokuka commented 4 years ago

Hey @pckeyan.

Can you please share your benchmark setup or a simple project which let me reproduce your issue | understand the setup behavior?

Cheers, Oleh

pckeyan commented 4 years ago

Hi @OlegDokuka,

I used directly into my business service, Let me try to peel it out. But from the environment standpoint, I used one Cassandra Docker container running on AWS EC2 - c5n.18xlarge. Same Backend Codebase to connect to Cassandra.

http Rest Call (API) - 800 TPS Non Blocking Http (API) - 1600 TPS RSocket(One Connection/Server) Connection Cache at Client - 300 TPS

I used the same code we tested on Issue 177 as mentioned above and I am using the Reactive Spring Cassandra with Datastax driver.

OlegDokuka commented 4 years ago

I'm asking because of different prefetch setup which may vary between reactive and blocking implementations in Spring.

Also, you point to a sample project, but it would be much better to have a simple client which can show RPS per a client

pckeyan commented 4 years ago

Definitely let me put together one for you to check. But if I remove the Cassandra and mock the data, I am getting ~5000 TPS. Something is blocking to Cassandra.

OlegDokuka commented 4 years ago

So, if you look at my slides on reactive database access, and in particular on the slide 33, which shows how reactive Cassandra works underneath, you may get an idea, why it may be slow, especially if backpressure is applied

pckeyan commented 4 years ago

Thank You, Let me go through the slide and will get back to you.

pckeyan commented 4 years ago

@OlegDokuka I tried to modify the fetchSize it has no effect. Let me try to get a sample for you to check.

OlegDokuka commented 4 years ago

@pckeyan any updates?

pckeyan commented 4 years ago

@OlegDokuka I tested with different parameters and I do not see any improvement. If you wish we can close this issue for now. I will reopen once I have a sample because I was testing this in my business application.

OlegDokuka commented 4 years ago

@pckeyan marking as need information. Feel free to provide a sample when it is ready

pckeyan commented 4 years ago

@OlegDokuka I added you as.a collaborator to my private projects. Please check this readMe for instructions. https://github.com/pckeyan/rsocket-load-balancer/blob/master/README.md. Please let me know your feedback.

OlegDokuka commented 4 years ago

@pckeyan looking

pckeyan commented 4 years ago

Hi @OlegDokuka, Any findings so far?

pckeyan commented 4 years ago

@OlegDokuka I removed my load balancer component and used LoadBalancedRSocketMono in my Client and directly called Server has increased the throughput to 980 transactions per second. So 3x increase if I remove the component. Any insights or findings? But still this is less than non-blocking http rate.

OlegDokuka commented 4 years ago

Hey. I still was not able to look due to hard work on the upcoming release, however, I'm not recommending to use a load-balanced RSocket for now.

I hope I will give you some updates shortly

pckeyan commented 4 years ago

Sure @OlegDokuka I Understand and will wait for your comments.

pckeyan commented 4 years ago

Just a gentle reminder @OlegDokuka

pckeyan commented 4 years ago

@OlegDokuka In the mean time, I tried with Postgresql. The RPS is at the same level of around 200-300. I tried with R2DBC but it is behaving weird. So could not complete the test with r2dbc. Can you please explain why the throughput is very low?

OlegDokuka commented 4 years ago

@pckeyan

do you mean HTTP vs RSocket on the pure JDBC is on the same level? If yes, I guess this can be simply DB limitation to have that number of requests per sec, I'm wondering what is the performance of the same without a network at all.

And is the second question related to jdbc vs r2dbc?

pckeyan commented 4 years ago

@OlegDokuka Either Cassandra or Postgresql, the throughput is same around 200 - 300 RPS. This is way less compared to HTTP (800 RPS), These testing is on same local device. The test uses JDBC. Cassandra states that it can support upto 80K RPS with One connection. For Postgresql, I increased the connections, but no success. Can you please test against a database and provide some metrics to compare?

R2dbc - When I tried with it, it is not even fetching data after a loop. I need to dig into it.

pckeyan commented 4 years ago

@OlegDokuka Any benchmark available against a database backend? Can you please share a sample? Please let me know your recommendations.

pckeyan commented 4 years ago

@OlegDokuka Any help here?