search

iconara / cql-rb

Cassandra CQL 3 binary protocol driver for Ruby
106 stars 31 forks source link

Make QueryResult lazily deserialize its rows #40

Closed iconara closed 10 years ago

iconara commented 11 years ago

As it is all frame deserialization happens in the IO thread, which is find for small frames, but for big results it's probably not a good idea. QueryResult/RowsResultResponse should be lazy and not deserialize anything until it's actually used). This would make it easier to support streaming in the future.

iconara commented 10 years ago

There's a LazyQueryResult in the protocol_v2 branch, and prepared statement results are now lazily decoded.

iconara commented 10 years ago

To clarify, this doesn't mean that frames are decoded in a streaming way, they are still buffered in memory, it's just that this chunk of memory is parsed in the client thread and not in the IO thread.

iconara commented 10 years ago

I think this is sufficiently implemented as of #74. Prepared statement results are lazily deserialized and you can page over large result sets. Streaming would be great, but that's for another release.