search

carrierwaveuploader / carrierwave

Classier solution for file uploads for Rails, Sinatra and other Ruby web frameworks
https://github.com/carrierwaveuploader/carrierwave
8.78k stars 1.66k forks source link

Carrierwave 3 extremely slow (retrieve_from_store! calling s3 over and over) #2697

Closed Amnesthesia closed 1 year ago

Amnesthesia commented 1 year ago

We're trying to figure out what's happened in version 3 of Carrierwave, where S3 is now so slow that it's virtually unusable.

Carrierwave seems to retrieve_from_store! as soon as you access #uploaders, and if a file doesn't exist on S3 (for whatever reason), it will keep retrying over and over and over, making it virtually unusable — but even when nothing is wrong, #store! seems to suffer from something similar. Uploading a < 500kb file takes over 10 seconds

Can't even access the uploader — even to get #identifier — without S3 being called and the file retrieved from store

Amnesthesia commented 1 year ago

Closing this because I submitted a pull request to fix this instead

wrmk commented 1 year ago

Faced the same problem: The application became very slow after upgrading to the 3 version.

Amnesthesia commented 1 year ago

@wrmk Have you managed to resolve it? and have you tried my PR out to see if that resolved it for you?

wrmk commented 1 year ago

@wrmk Have you managed to resolve it? and have you tried my PR out to see if that resolved it for you?

We downgraded to the 2 version. Have no possibility to investigate how to resolve or try your PR.

rajyan commented 1 year ago

I think this issue might be releated https://github.com/fog/fog-aws/issues/690 could someone confirm whether the latest master of fog-aws eases this issue?

geemus commented 1 year ago

fog-aws v3.21.0 has fixes which seem related (it should help with the retries)

Amnesthesia commented 1 year ago

@geemus Thank you! We just found that this was all caused by fog and solved it by replacing fog with carrierwave-aws instead

geemus commented 1 year ago

@Amnesthesia sorry to hear you aren't using fog any more, but glad your problem is solved.