clue / reactphp-sqlite

Async SQLite database, lightweight non-blocking process wrapper around file-based database extension (ext-sqlite3), built on top of ReactPHP.
https://clue.engineering/2019/introducing-reactphp-sqlite
MIT License
51 stars 10 forks source link

Support streaming result sets #19

Open clue opened 5 years ago

clue commented 5 years ago

Currently, the API only supports buffering the whole result set in memory. This makes sense for smaller result sets as working with an in-memory data structure with some dozens or hundreds of records works perfectly fine.

However, when processing larger result sets with hundreds or thousands of records, it would make sense to use a streaming approach so that one is no longer limited by memory in how many records can be processed.

Eventual API will use ReactPHP's existing stream API and could look something like this:

$stream = $db->queryStream('SELECT * FROM users');

$stream->on('data', function ($row) {
    var_dump($row);
});
$stream->on('end', function () {
    echo 'DONE' . PHP_EOL;
});

Refs https://github.com/friends-of-reactphp/mysql/issues/50