Closed dannymandel closed 1 month ago
We should investigate the https://fastapi.tiangolo.com/advanced/middleware/#gzipmiddleware for use in the download service. The files themselves are quite large and it would save a lot of download time since the jsonl files should compress nicely.
We should investigate the https://fastapi.tiangolo.com/advanced/middleware/#gzipmiddleware for use in the download service. The files themselves are quite large and it would save a lot of download time since the jsonl files should compress nicely.