search

Crowdtilt / CrowdtiltOpen

Open source crowdfunding platform -
http://open.tilt.com
Other
598 stars 314 forks source link

Delayed job for image uploads #123

Open mattlebel opened 10 years ago

mattlebel commented 10 years ago

Right now, there is the potential for image uploads to timeout (>30s to process) and cause an application error.

We should move the upload to a delayed job using this gem (https://github.com/jrgifford/delayed_paperclip/) or something similar. This introduces some dependencies, including Resque / Redis, but will increase the robustness of all uploads (images for now, possibly videos in the future).

Note that this fix would require running at least one worker dyno on your Heroku application to handle background tasks.

msaint commented 10 years ago

I think you'll find a number of tasks that are better suited for background jobs -- their addition to the crowdhoster app is inevitable. From what I've seen Sidekiq (https://github.com/mperham/sidekiq) is a very popular route to go.

For dealing with image uploads specifically, another option is to use what Crowdtilt uses - https://www.inkfilepicker.com/ - which completely bypasses the app to upload images directly to S3, then provides a url param interface for processing the image to whatever size you need. Pretty nice. But does introduce another 3rd party service dependency.

An alternative "DIY" way to achieve what Filepicker does would be to use something like this gem: https://github.com/dwilkie/carrierwave_direct

mattlebel commented 10 years ago

@msaint Awesome alternatives here! There exists a Railscasts for Sidekiq as well: http://railscasts.com/episodes/366-sidekiq

msaint commented 10 years ago

yep :) you'll find an implementation of it (guided by that very railscast) on a little project called EntranceJams...

msaint commented 10 years ago

Been dabbling with Cloudinary lately for client side image uploads...pretty sweet and their free plan is incredibly generous and fully capable (in contrast to a service like Ink Filepicker, which is not so generous and won't even do conversions on the free plan). Given that Crowdhoster already has a few "freemium" dependencies, namely S3, it may be worth considering a switch over to Cloudinary for images and killing the S3 dependency altogether. Asset Sync is a rather fickle (and time consuming) gem, and Heroku (as well as amazon) no longer recommend treating S3 as a CDN. If the app by default simply served its own static assets, adding a real CDN like Cloudfront would be an almost trivial task, especially on Heroku with an add-on like CDN Sumo.