elixir-ecto / db_connection

Database connection behaviour
http://hexdocs.pm/db_connection/DBConnection.html
306 stars 113 forks source link

Introduce prepare_stream and stream using cursors #66

Closed fishcakez closed 7 years ago

fishcakez commented 7 years ago

Adds prepare_stream and stream that stream results using new callbacks handle_open/4, handle_fetch/4, handle_close/4.

The prepare phase of prepare_stream follows the same rules as prepare_execute: close query on describe or encode error. Other errors do not cause a close.

For the cursor handle_close/4 is not called if handle_fetch/4 returns a :done tuple, it is assumed handle_fetch/4 did any required cleanup. Otherwise cursor is closed.

The cursor is a constant throughout the stream because it may not be possible to recover a change in cursor if an error occurs. As a query should not be executed in handle_open/4, but started in `handle_fetch/4, cursors may need to be tracked in the callback module.

fishcakez commented 7 years ago

Dropped support for streams outside transactions.

josevalim commented 7 years ago

FWIW, I took a cursory glance and this looks good to go!

fishcakez commented 7 years ago

Some refactoring of the callbacks to always cleanup the cursor explicitly and to allow differentiating "first" cursor call, which does the execute, from subsequent "next" cursor calls. New names should be closer to the SQL cursor equivalents (handle_first does open and fetch first, handle_next does fetch next, handle_deallocate maybe closes and deallocates). This pattern seems a closer match to the wire protocols of other databases. It will also show where the execute (first) occurs in the logs.