UCL-ARC / museum_of_dreams

Repository for the Museum of Dreamworlds Project
https://museumofdreamworlds.org
0 stars 0 forks source link

optimise large file upload #69

Closed acholyn closed 2 months ago

acholyn commented 3 months ago

Whilst the nginx upload limit is 3gb, in practice it takes far too long to upload even 900mb so this needs to be handled differently - s3 multipart uploads or presigned urls is probably the way to go It may also be due to us using t3.micro instances, we might need to use something with more processing power or reconfigure the scaling

acholyn commented 2 months ago

As we're opting for mediacentral to house the videos, we don't need to implement this on our side anymore. There is an upload limit of 3gb but the procedure has not been optimised. This limit should be sufficient but can be reduced if necessary.