search

energy-data / energydata.info

energydata.info - open data and analytics for a sustainable energy future
http://energydata.info
MIT License
26 stars 6 forks source link

File size upload increase #366

Open gclapp1 opened 1 year ago

gclapp1 commented 1 year ago
  1. Routing uploads to S3 instead of locally
  2. Trial of AWS Multipart Uploads service
jodiegardiner commented 1 year ago

s3storage extension is already attached for storing tiles

jodiegardiner commented 1 year ago

now i just need to point the filestore at it. Multipart Upload configuration required

gclapp1 commented 1 year ago

@jodiegardiner checking in on this and #384

jodiegardiner commented 1 year ago

Sorry Grayson, this is working now and has been for a while! My bad.

jodiegardiner commented 1 year ago

Hi Grayson, I've closed this one in error. Don't know what I was responding to. Checking in with my team to give you a proper update this morning.

jodiegardiner commented 1 year ago

We created an extension called s3filestore and have tested on staging. To proceed with this going forward, we'll need to move all currently locally hosted (on linode) resources to s3 and from then on, newly uploaded resources will go to s3.

It still remains to be seen if this is the best solution for the actual problem at hand which is file size limitations (and total space limitations) in the current system.

gclapp1 commented 1 year ago

Great to hear the updates from yesterday-please feel free to close ticket once live