search

libsql / sqld

LibSQL with extended capabilities like HTTP protocol, replication, and more.
https://libsql.org
900 stars 38 forks source link

Limits on payload size prevents taking db exports #637

Open avinassh opened 1 year ago

avinassh commented 1 year ago

A user in discord was not able to take export:

turso db shell mighty-night .dump > my_db.sql
Error: failed to execute SQL: SELECT * FROM 'users'
Response is too large

discord - https://discord.com/channels/933071162680958986/1146289112970707014

I believe it's due to the limits we have set on the payload size. I think we should add a better mechanism to take exports

CodingDoug commented 1 year ago

The .dump command implemented in libsql-shell-go could internally paginate results from the queries it issues.

MarinPostma commented 1 year ago

We should expose an endpoint for dumps that streams the dump instead. The .dump mega command should call that endpoint