search

planetscale / database-js

A Fetch API-compatible PlanetScale database driver
https://planetscale.com/docs/tutorials/planetscale-serverless-driver
Apache License 2.0
1.17k stars 35 forks source link

Explore supporting StreamExecute/OLAP workloads #55

Open mattrobenolt opened 2 years ago

mattrobenolt commented 2 years ago

Within some contexts, especially serverless, the memory limits are very low in process. So it can be desirable when performing aggregations in process, or if also streaming out to the end user, to be able to leverage StreamExecute from our API.

I will say, this is going to be rather non-trivial, but I'd like us to see what it'd take to support without fully adopting connect-web or protobufs.

Streaming requires adopting the Connect protocol, rather than simply application/json, which can be done, and is done within connect-web. https://connect.build/docs/protocol#streaming-response

To add to this, this feature is required to support OLAP queries.

cc @reconbot

mattrobenolt commented 2 years ago

This is a functioning POC https://gist.github.com/mattrobenolt/6ca8c37185578cac8970de5a82dcfd52

It definitely needs more work. I also talked with the buf folks, and they agree that we should vendor in their codec stuff like I'm doing here.

sabin26 commented 2 months ago

I need to use streaming api with Cloudflare Workers. Mysql drivers that support streaming are not compatible with workers. Is there any plan to support this ?