Add support for results streaming

databricks / databricks-sql-nodejs

Databricks SQL Connector for Node.js

Apache License 2.0

23 stars 32 forks source link

Add support for results streaming #251

Open ShantiCumulio opened 3 months ago

ShantiCumulio commented 3 months ago

I've built a plugin for connecting Databricks to Luzmo. I would like to be able to process the results from a query as a readable stream, similar to how this library creates for Postgres to speed up query + data processing times.

kravets-levko commented 3 months ago

Hi @ShantiCumulio! This feature is already in our backlog, but, unfortunately, currently there are no estimates for when I can work on it. If you want to volunteer on implementing it - you're welcome 🙂

kravets-levko commented 3 months ago

A quick update on this issue. I was working on adding an iterable interface to operation (databricks/databricks-sql-nodejs#252) when I realized that Node has a built-in helper to convert iterators into readable streams. So expect streams support very soon