AssetSync / asset_sync

Synchronises Assets between Rails and S3
1.88k stars 346 forks source link

Works with Rails 3.2.0? #41

Closed manuelmeurer closed 12 years ago

manuelmeurer commented 12 years ago

I just upgraded to Rails 3.2.0 and asset_sync 0.2.5 and it doesn't seem to work. I followed the upgrade instructions in the Readme and everything went fine, but when I deploy with Capistrano, the assets are not uploaded to S3.

This is my config/initializers/asset_sync.rb:

AssetSync.configure do |config|
  config.fog_provider          = 'AWS'
  config.aws_access_key_id     = AppConfig.aws[:key]
  config.aws_secret_access_key = AppConfig.aws[:secret]
  config.fog_directory         = AppConfig.aws[:bucket]

  # Increase upload performance by configuring your region
  config.fog_region = AppConfig.aws[:region]

  # Don't delete files from the store
  config.existing_remote_files = 'delete'

  # Automatically replace files with their equivalent gzip compressed version
  config.gzip_compression = true

  # Use the Rails generated 'manifest.yml' file to produce the list of files to
  # upload instead of searching the assets directory.
  # config.manifest = true

  # Fail silently. Useful for environments such as Heroku
  # config.fail_silently = true
end

and my lib/tasks/asset_sync.rake:

Rake::Task['assets:precompile'].enhance do
  AssetSync.sync
end

When I call AssetSync.sync from the Rails console on production after deploy, it works as expected and the assets are uploaded.

kpfefferle commented 12 years ago

I had a similar issue after upgrade to 3.2.0 where assets will sync and upload if I compile them locally, but do not upload on push to Heroku.

maioral commented 12 years ago

+1 in heroku too.

aaronchi commented 12 years ago

appears not to be running anymore under Rails 3.2/heroku :(

mrship commented 12 years ago

I'm having the same issue on Heroku.

My workaround is to run AssetSync.sync from the heroku console after deploy.

subblue commented 12 years ago

Same issue here. Currently getting around it by running heroku run rake assets:precompile manually after deployment to Heroku.

davidjrice commented 12 years ago

Damn, I thought @manuelmeurer's issue was down to not integrating capistrano. Turns out there is an issue with Rails 3.2 and Heroku. I don't fully understand what's going on here yet.

If you add Rake.application.options.trace = true in application.rb it gives us some extra rake debugging in a Rails app.

On a git push heroku it would appear that the rake assets:precompile task does not get "enhanced" correctly and subsequently not run. However running heroku run rake assets:precompile after deploy does work.

Rails 3.2 appears to be doing some more crazy stuff now to invoke rake tasks, which appear to wipe out the "enhance" of rake assets:precompile

As a work around the following appears to work for me (on Heroku at least). If you are to create the file _lib/tasks/assetsync.rake and add this task.

Rake::Task["assets:precompile:nondigest"].enhance do
  Rake::Task["assets:environment"].invoke if Rake::Task.task_defined?("assets:environment")
  AssetSync.sync
end

You may notice this is the rake task from within the asset_sync gem but with one key difference, we are now "enhancing" the rake assets:precompile:nondigest task. We may consider making this the standard approach going forward if it works for people...

If it works for Rails 3.1.x apps too, that'll be great.

manuelmeurer commented 12 years ago

This seems to work in both Rails 3.1.3 and 3.2.0.

davidjrice commented 12 years ago

Great, I've created a branch and improved the default asset_sync task. If you want to get this working you can now just use the following in your Gemfile. I want to see if I can make sure it works from Rails 3.1.0 upwards if possible before releasing.

 gem 'asset_sync', :git => 'git://github.com/rumblelabs/asset_sync.git', :branch => 'rails-3-2'
davidjrice commented 12 years ago

Okay, that's merged into master and released in Version 0.2.7

manuelmeurer commented 12 years ago

Sweet, thanks!

amalagaura commented 12 years ago

It still does not upload assets for me in Rails 3.2. I have upgraded to 0.2.7. The rake task does work though.

It's a standard setup. I can post it and logs, but is nothing special. The files just don't get uploaded to S3 until I run the extra rake task.

iainad commented 12 years ago

I have the same issue with not uploading assets (Rails 3.2/asset_sync 0.2.7). Manually calling AssetSync.sync from the console works fine, but it's not called post asset:precompile.

subblue commented 12 years ago

Still having the same problem here with 0.2.7 too

maioral commented 12 years ago

Still having the problem in 0.2.7 too.

davidjrice commented 12 years ago

@leandromaioral @subblue @amalagaura @iainad if you need any further help can you create a new issue. The library definitely works with Rails 3.2 therefore it's likely it is something else.

If you have not enabled the Heroku _user_envcompile feature from Heroku labs then asset_sync will not actually sync on git push heroku without running the rake task after OR by hardcoding all the config in your initializer/yml.

amalagaura commented 12 years ago

@davidjrice I have my keys set via AssetSync.configure in an initializer. I think the issue is that now I had to turn on: config.assets.initialize_on_precompile = false

because the precompile was failing in 3.2, but not in 3.1. In 3.1 I didn't have that command working so there was a workaround that disabled after some parts of the initialization.

On the README it states:

Will not work on heroku using ENV variables with the configuration as described below, you must hardcode all variables

However are you saying it will work with the user_env_compile feature? Or I can simply use a YML instead of the .rb initializer so the YML wil get detected?

davidjrice commented 12 years ago

@amalagaura so does it work now?

yes, you can use ENV variables with the user_env_compile labs add on. As these are now exposed to the ruby buildpack, they are instantiated when rake assets:precompile runs.

It should work with the initialize_on_precompile flag set to true or false.

amalagaura commented 12 years ago

Still isn't working. First I tried with the _user_envcompile by using the YML with string interpolation of the ENV variables. That didn't work, so I tried hardcoding the access keys in the YML. Also didn't work.

I added config/asset_sync.yml which I copied from the readme.. It doesn't throw any error message, just skips the upload.

Now trying the upload via rake that also breaks. This must be some other issue. It was working with the .rb initalizer via rake. Now I get Fog provider can't be blank, Fog directory can't be blank

But my YML is exactly according to the README.

defaults: &defaults
  fog_provider: 'AWS'
  fog_directory: 'sales-system-assets'
  aws_access_key_id: 'ACCESS'
  aws_secret_access_key: 'SECRET'
  fail_silently = false

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults
amalagaura commented 12 years ago

So I readded my initializer, and the rake task works, but the upload on deploy still fails silently even though I have fail_silently as false. That suggests the rake enhance is not working. With YML both fail. All on Cedar stack of course.

davidjrice commented 12 years ago

Is config.assets.digest set to true?

Sent from my tiny Internet machine

On 27 Jan 2012, at 17:18, Ankur Sethi reply@reply.github.com wrote:

So I readded my initializer, and the rake task works, but the upload on deploy still fails silently even though I have fail_silently as false. That suggests the rake enhance is not working. With YML both fail. All on Cedar stack of course.


Reply to this email directly or view it on GitHub: https://github.com/rumblelabs/asset_sync/issues/41#issuecomment-3690345

amalagaura commented 12 years ago

Yes in config/environments/production.rb

davidjrice commented 12 years ago

Hmm... If you can share the app with me I'll have a closer look

Sent from my tiny Internet machine

On 27 Jan 2012, at 18:31, Ankur Sethi reply@reply.github.com wrote:

Yes in config/environments/production.rb


Reply to this email directly or view it on GitHub: https://github.com/rumblelabs/asset_sync/issues/41#issuecomment-3691644

amalagaura commented 12 years ago

@davidjrice I sent an email to the contact address on your website

iainad commented 12 years ago

Just fyi, here's my rake trace showing that asset_sync doesn't get called:

  * executing `deploy:assets:precompile'
  * executing "cd /webapps/site-preview/releases/20120127205114 && bundle exec rake RAILS_ENV=preview RAILS_GROUPS=assets assets:precompile"
    servers: ["site.domain.tld"]
    [site.domain.tld] executing command
*** [err :: site.domain.tld] DEPRECATION WARNING: Calling set_table_name is deprecated. Please use `self.table_name = 'the_name'` instead. (called from require at /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler/runtime.rb:68)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:precompile (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:precompile
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:precompile:all (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:precompile:all
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:precompile:primary (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:environment (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:environment
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke environment (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute environment
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke tmp:cache:clear (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute tmp:cache:clear
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:precompile:primary
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] DEPRECATION WARNING: Calling set_table_name is deprecated. Please use `self.table_name = 'the_name'` instead. (called from require at /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler/runtime.rb:68)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:precompile:nondigest (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke assets:environment (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:environment
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke environment (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute environment
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Invoke tmp:cache:clear (first_time)
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute tmp:cache:clear
*** [err :: site.domain.tld] 
*** [err :: site.domain.tld] ** Execute assets:precompile:nondigest
*** [err :: site.domain.tld] 
    command finished in 242673ms
  * executing `deploy:symlink'

I have config.assets.digest set to true as above too with rails 3.2 and asset_sync 0.2.7.

amalagaura commented 12 years ago

@leandromaioral @subblue @davidjrice David has been looking at my app. So far a custom task in lib/tasks/asset_sync.rake is working to enhance the rake task.

Rake::Task["assets:precompile:nondigest"].enhance do
  Rake::Task["assets:environment"].invoke if Rake::Task.task_defined?("assets:environment")
  AssetSync.sync
end

David didn't know why my particular Rails 3.2.1 app was different from others and needed the rake task. He said he's looking into it. This is a workaround I suppose.

@iainad Your issue is different because your explicitly run rake task is not uploading. You have RAILS_ENV=preview. Maybe that is the issue. The README says only production env works on Heroku, I don't know if it is supposed to work if you are running locally.

iainad commented 12 years ago

@amalagaura actually I'm not using Heroku (just asset_sync on my normal rails deploy to EC2).

Anyways, adding that as a rake task to my lib/tasks dir in my app works just fine so that looks to be a sensible approach. I'd worked around it using a similar task called post precompile, but this is cleaner and easier.

So works for me too!

amalagaura commented 12 years ago

@iainad Well looks to be 2 different but related issues. My issue was that the Heroku push did not upload assets. A local or heroku rake task was uploading. @davidjrice was looking at that on my app.

Your issue and the original issue by @manuelmeurer was that the assets:precompile rake tasks don't upload. So in both cases the rake task in the gem is not being picked up by the application.

masterkain commented 12 years ago

Same issue here, not on Heroku but asset_sync is never called nonetheless.

nextofsearch commented 12 years ago

Rails 3.2.1 with cedar stack on Heroku. labs plugin installed.

@amalagaura the custom task you put on the above doesn't work for me. :(

ekampp commented 12 years ago

It does say in the read me for asset_sync that you need to run the assets:precompile after pushing to Heroku.

So basically I run git push heroku master and when that is done I run heroku run rake assets:precompile. This will ensure that heroku picks up the fact that I would like to precompile my assets with asset_sync.

This works excellently for me.

amalagaura commented 12 years ago

It's supposed to work without running an extra rake task because heroku runs the assets:precompile step as part of a deploy. After @davidjrice looked into it and added the extra code it is working on mine. But apparently that fix is not working for everyone.

ekampp commented 12 years ago

Ahh.. I see :) Well, that rake task that you referenced works awesome for me. =)

davidjrice commented 12 years ago

Everyone. This is a sporadic issue where Rails 3.2 is not seeing tasks registered in a Gem when run under the rake assets:precompile task. The reason it is sporadic is, in attempting to debug this it seems to happen in some projects and not others.

We're working to find a permanent solution that will make the gems enhancements to the precompile tasks visible again. Until then you can simply put the following in lib/tasks/asset_sync.rake.

Rake::Task['assets:precompile'].enhance do
  AssetSync.sync
end

Depending on your configuration, you may need to enhance the rake assets:precompile:nondigest

Rake::Task["assets:precompile:nondigest"].enhance do
  AssetSync.sync
end

Running rake assets:precompile in Rails 3.2 is SLOW. AssetSync does add to that, however the majority of time is spent in Rails.

seivan commented 12 years ago

If that doesn't help. Try to run bundle exec rails runner "AssetSync.sync" on the heroku server after pushing.

git push heroku master heroku run bundle exec rails runner "AssetSync.sync"

davidjrice commented 12 years ago

@seivan that step is not required. The official solution is to define your own rake task as described above.

Ensuring that your heroku configuration variables are correct and that you have enabled the user_env_compile feature from heroku labs.

Using heroku run bundle exec rails runner "AssetSync.sync" will mean your application is deployed and the assets are not synced to your asset host. This could cause issues with assets not being available when the new version of your application comes online.

seivan commented 12 years ago

@davidjrice I understand. So far defining the rake tasks with enhance does not work. Changed some javascript, pushed to Heroku. It could not find it on s3. Ran heroku run bundle exec rails runner "AssetSync.sync" and all is good. I have to run heroku run bundle exec rails runner "AssetSync.sync" after each deploy, even with enhanced rake tasks.

davidjrice commented 12 years ago

Okay everyone please checkout the latest Version 0.2.12 I've made some changes to how asset_sync gets called.

I have tested with both Rails 3.1.1 and Rails 3.2. However I'd appreciate hearing back if this doesn't work for anyone. To test this:

I am going to close this issue, if there is anything additional that comes up because of this upgrade, please start a new issue as this one is getting quite long!

Check out my commit dfccf31 if you're interested on what was causing the issue.

amalagaura commented 12 years ago

I just tested and I can confirm it works on Heroku. Rails 3.2.2 with ruby 1.9.3 ( I deleted the extra rake task too)

ehoch commented 12 years ago

Fixed for me as well! Rails 3.2.1 with ruby 1.9.2. Thanks!!

skwp commented 11 years ago

I'm having a complete mystery issue where one of the developers on my team when he runs deploy the asset sync task is not invoked. We compared rake/cap/asset_sync/codebase versions and everything is identical. When we look at the output of his deploy, the asset sync is simply not invoked. We tried the custom rake task but still to no avail. Has anyone seen this issue recently? Asset sync 0.5.4 rails 3.2