vert-x3 / vertx-mysql-postgresql-client

This client is deprecated - use instead
https://github.com/eclipse-vertx/vertx-sql-client
Apache License 2.0
117 stars 59 forks source link

All the result of SQLRowStream run in a loop #148

Closed okou19900722 closed 1 year ago

okou19900722 commented 5 years ago

https://github.com/vert-x3/vertx-mysql-postgresql-client/blob/cbdbf729ad327834211140112b7958807a449650/vertx-mysql-postgresql-client-jasync/src/main/java/io/vertx/ext/asyncsql/impl/AsyncSQLRowStream.java#L69-L74

https://github.com/vert-x3/vertx-mysql-postgresql-client/blob/cbdbf729ad327834211140112b7958807a449650/vertx-mysql-postgresql-client-jasync/src/main/java/io/vertx/ext/asyncsql/impl/AsyncSQLRowStream.java#L93-L96

it will fetch all the result and never pause until it's finish

okou19900722 commented 5 years ago

128 is the default limit of fetch in vertx-jdbc-client

https://github.com/vert-x3/vertx-jdbc-client/blob/cd3d609236c485221e79021df7a41b885d66af6f/src/main/java/io/vertx/ext/jdbc/impl/actions/StreamQuery.java#L36

oshai commented 5 years ago

In the driver itself, there is no such concept of FETCH_SIZE. Instead, all results are read to memory before future completes (or callback is called). So the parameter has no real effect as far as I know.

okou19900722 commented 5 years ago

but it will block the event loop

oshai commented 5 years ago

do you mean that user code will block the event loop?

okou19900722 commented 5 years ago

one million row will fetch in a loop, it's will block the event loop. I can only manually pause, and wait for 1 millisecond before resume

oshai commented 5 years ago

Another alternative is to schedule it later (manually) with delay. However, I am not sure there is an actual performance impact on blocking the event loop with cpu bound work. Usually, servers are io bound and not cpu bound and the pool of threads for events loop is of the size of the number of cores.

vietj commented 5 years ago

if you are using Postgres have you tried using the Reactive PG Client that provides a similar concept using cursor ?

okou19900722 commented 5 years ago

I'm working with mysql. Looking forward to vertx-sql-client