simonw / datasette

An open source multi-tool for exploring and publishing data
https://datasette.io
Apache License 2.0
9.61k stars 690 forks source link

if csv export is truncated in non streaming mode set informative response header #1553

Open fgregg opened 2 years ago

fgregg commented 2 years ago

streaming mode is currently not enabled for custom queries, so the queries will be truncated to max row limit.

it would be great if a response is truncated that an header signalling that was set in the header.

i need to write some pagination code for getting full results back for a custom query and it would make the code much better if i could reliably known when there is nothing more to limit/offset

fgregg commented 2 years ago

would also be good if the header said the what the max row limit was

fgregg commented 2 years ago

these headers would also be relevant for json exports of custom queries

simonw commented 2 years ago

This is a really interesting idea - kind of similar to how many APIs include custom HTTP headers informing of rate-limits.