search

eyra / mono

Next platform repo
https://eyra.co
GNU Affero General Public License v3.0
7 stars 4 forks source link

[Storage] CMS: export all files #892

Closed mellelieuwes closed 2 months ago

mellelieuwes commented 3 months ago

Export files is implemented using Packmatic: https://github.com/evadne/packmatic This means we stream the zip file without using the filesystem or having the whole zip in memory. Every file is downloaded and streamed on the fly.

mellelieuwes commented 2 months ago

@TjerkNan we need to test how the implementation is behaviing with a lot of files (1000+) and bytes (large files). To test the storage export the type of files in the storage do not matter (no json required). So for testing you can upload big files (images, zips, etc) to test the user experience. For inserting a large amount files we might need to create a script. Let me know if you need help with this.

TjerkNan commented 2 months ago

I've created the script to generate files in S3 of various sizes and amounts so I think this particular card is solved.