search

nulib / arch

Northwestern University institutional repository, built on Samvera's Hyrax gem.
2 stars 0 forks source link

Increase the max file size to 2gb #170

Closed chrisdaaz closed 6 years ago

chrisdaaz commented 7 years ago

*Waiting on 2.0 upgrade

The Sufia default for maximum file size for self-deposits is 500 MB

I believe this can be configured in the uploader.js file: https://github.com/projecthydra/sufia/blob/master/app/assets/javascripts/sufia/uploader.js#L32

DigitalHub accepts maximum file sizes of 2GB, which, I believe, is the limit for HTTP uploads: https://digitalhub.northwestern.edu/help/#Uploading Files to DigitalHub

What if we increased the file size to 2 GB? Is it simply a matter of changing the code in this file, or is there a reason why we wouldn't want to increase the size limit?

DONE:

davidschober commented 7 years ago

@bmquinn and @csyversen and @kdid is this a fast fix? (configuration?)

csyversen commented 7 years ago

@d-venckus and/or @Toputnal , we'll probably have to work with you guys to make sure the file system can handle lots of files

Toputnal commented 7 years ago

@csyversen are these files being stored on the Isilon? If so, a large number of files that are up to 2GB each should not be a problem.

csyversen commented 7 years ago

@Toputnal yeah, the originals will be stored in arch's fedora 4 instance and the derivatives will be stored in /var/www/nufia/shared/tmp/derivatives/, so i think thats isilon. can you double check that for me though?

Toputnal commented 7 years ago

Is this on nufiaweb-p or on nufiarepo--p, @csyversen ?

csyversen commented 7 years ago

@Toputnal web-p!

d-venckus commented 7 years ago

I will investigate moving the derivitaves folder to Isilon.

d-venckus commented 7 years ago

We'll also need to address a different backup strategy than merely using Veeam backups (28 days only) - moving to Isilon should suffice.

csyversen commented 7 years ago

awesome, thanks @d-venckus !

i'll go ahead and up the file size limit on my end!

csyversen commented 7 years ago

just to give an update on this, this required a change in upstream hyrax. I made that change and its going to get reviewed and hopefully wrapped up in a release soon-ish. After that we can upgrade hyrax in Arch, make our local changes (which i've already done), and we can test it out!

egspoony commented 7 years ago

Removed devops label.

davidschober commented 7 years ago

@csyversen waiting on this should be OK, but we need to move on #179.

kdid commented 7 years ago

TODO: once we upgrade to Hyrax 2.0 make sure the setting is configured.

kdid commented 6 years ago

@chrisdaaz - the limit is increased. On the larger files there is still a bit of time lag after the green progress bar is at 100% before it finishes. For me a 600MB file took about +30 seconds and a 1GB file about a +minute. Let us know your experience this during testing and whether this is in the range of acceptable.