Asynchronous processing

r-dbi / dbi3

DBI revisited

https://r-dbi.github.io/dbi3

37 stars 2 forks source link

Asynchronous processing #33

Closed krlmlr closed 4 months ago

krlmlr commented 4 years ago

[ ] Connection (to allow aborts)
[ ] Query execution
[x] Fetching (should test % 1024 instead of % 1000, )

Caveats

MariaDB and MySQL interface is different: https://mariadb.com/kb/en/using-the-non-blocking-library/
Only Connector/C (MariaDB) seems to explicitly support termination
Use Rcpp::checkUserInterrupt() in a try {} block
Try extern void Rsleep(...) for accessing a portable sleep() function, otherwise call Sys.sleep() from C

krlmlr commented 2 years ago

To be really useful, e.g. for Shiny apps, we'd need to support promises. With Arrow, the entire processing of a query can run in a separate thread (e.g. a {later} background task, https://github.com/r-lib/later), and signal to the main R thread when it's done.

krlmlr commented 4 months ago

This is a blend of #19 and #20, closing.