search

codenotary / immudb

immudb - immutable database based on zero trust, SQL/Key-Value/Document model, tamperproof, data change history
https://immudb.io
Other
8.52k stars 337 forks source link

Support streaming response in SQLQuery/TxSQLQuery rpcs #1967

Closed ostafen closed 2 months ago

ostafen commented 2 months ago

This PR makes methods SQLQuery/TxSQLQuery return a streaming response , thus removing the present limitation on the maximum number of rows returned by a query. From the postgres server side, streaming has been implemented by splittings the result in multiple messages, according to the pg protocol spec).

To make this change backward compatible and ease the transition of existing grpc clients, a field has been added to the SQLQueryRequest message: acceptStream (such field can be deprecated and finally removed when all clients will be migrated). Such field is used by the server to establish if streaming should be used or if an error should be returned to the client when the number of rows to return exceeds the limit. Since existing clients are not aware of the new field, it will default to false in their requests, so the behavior will be exactly the same as before.

Nothing is required for pg clients, since they are already designed to handle streaming.

NOTE: The method has been checked with the following SDK: Python, Java and NodeJS.

coveralls commented 2 months ago

Coverage Status

coverage: 89.404% (-0.003%) from 89.407% when pulling 59ad53c6956fdbb9872d7a66731bb815c2129ea4 on ostafen:feature/sql-streaming into e56001c423fa301295cac1919e55aaa8bfa1bf0c on codenotary:master.