pglombardo / PasswordPusher

🔐 Securely share sensitive information with automatic expiration & deletion after a set number of views or duration. Track who, what and when with full audit logs.
https://docs.pwpush.com
Apache License 2.0
1.94k stars 339 forks source link

Multi-part Uploads capability #1895

Open kayvanaarssen opened 6 months ago

kayvanaarssen commented 6 months ago

When needing to Push 5+ GB Files. You would want Multi-Part uploads Its also recommended to do this with files larger than 100MB

Wasabi:

What are the minimum and maximum object sizes that can be stored in Wasabi. Individual objects stored in Wasabi can range in size from a minimum of zero bytes to a maximum of 5 terabytes. The largest object that can be uploaded in a single HTTP PUT is 5 gigabytes. For objects larger than 100 megabytes, Wasabi recommends customers consider using the multi-part upload capability. The total volume of data and number of objects you can store with Wasabi is unlimited (note that the free trial is capped at 1 TB).

DigitalOcean:

Object and File Size Limits Buckets have the following file size limits:

PUT requests can be at most 5 GB.

Each part of a multi-part upload can be at most 5 GB.

Each part of a multi-part upload must be at least 5 MiB, except for the final part.

Multi-part uploads can have at most 10,000 parts.

The maximum supported total size of a multi-part upload is 5 TB.

PUT requests and individual parts of multi-part uploads sent to the Spaces CDN using pre-signed URLs have a maximum payload of 8,100 KiB, or 7.91 MiB.

https://docs.digitalocean.com/products/spaces/details/limits/#:~:text=The%20maximum%20supported%20total%20size,8%2C100%20KiB%2C%20or%207.91%20MiB.

pglombardo commented 6 months ago

I think the problem pwpush has in ActiveStorage. It seems to have multipart support which means the issue maybe in the ActiveStorage JS library.

This is on my list to investigate/resolve.

I agree though - this needs to be added/fixed.

kayvanaarssen commented 6 months ago

Thanks, let me know if I need to test something!