Feature: automatically update cached files

[ari]

Rails' cache hander suffers from an enormous problem: the joined and cached files which smurf so nicely compresses are never updated unless you delete them by hand. Quite impossible to remember to do every time on a production server of course.

Would smurf be able to weave more magic to determine whether the date stamp of any file which is being combined is later than the combined file and then regenerate the combined, minified output?

[thumblemonks]

Hey Ari,

Thanks for the feedback. Can I get a little more info about this issue from you? Just so I understand, you are using Smurf in a production environment and on the first go (when no files are compressed/cached) Smurf does what you expect. Then you redeploy your app, but the previously compressed/cached files are not removed causing the JS/CSS/whatever to be inaccurate?

Let us know and we're on it. Also, what version of Rails are you on (just so we get the whole picture)?

Cheers,

• gus

[ari]

http://github.com/ari/clockingit is the project if you want to see it in more detail (task and project management system).

Rails 2.3.5 and yes, your summary is accurate. I took a look at the code in Rails (even in the trunk of rails development on github) and there is nothing in there which would even attempt to detect that assets have been updated.

It seems fairly trivial: look at the modified dates on all the relevant files to decide whether a rebuild is needed. So just touching one of the source files should cause a rebuild.

Just to be clear, this isn't a bug in Smurf. But it is a major shortcoming in Rails asset management.

I understand that checking the dates on a dozen files for every request is probably too time consuming, so I'd be happy with something sensible, like the first time the application is run (so the user needs to restart their production application to get changes), or (even better) once a minute or so. I'm a Java person, not Ruby, so I'm not really sure where to start: all this arbitrary overriding of classes in Ruby makes me want to run a mile :-)

[ari]

http://github.com/rails/rails/blob/9415935902f120a9bac0bfce7129725a0db38ed3/actionpack/lib/action_view/helpers/asset_tag_helper.rb at around line 285

That conditional doesn't even make sense. If not (caching && output exists) then rewrite output. Huh? But definitely no date checking in there.

[ari]

Any thoughts about whether this is doable, or should I pursue some other approach to expiring the cached files?

[thumblemonks]

Dang it. I became "that" repository owner. The one that sits on things and doesn't respond. Sorry about that. I've been a bit busy lately.

Ok, let's see. I remember this problem being really annoying to me waaaaay back in Rails 2.1.something. I've been able to avoid the issue in production simply because the deployment script (and I always use Vlad the Deployer) creates a new directory when it checks out the latest code on deployment. Which means, cached files "go away" on each deploy.

To fix the issue could be done in a few ways, all of which you suggested: remove cached files when rails boots, compare dates on files, et al. It's definitely a Rails fix and being such, it seems worthwhile to add this as a Rails option in the configurator (ActionController::Base.expunge_cached_files_on_boot or ActionController::Base.expire_expunged_cache_files). I also don't know what Rails 3 stuff looks like (may break Smurf altogether).

I need to look into this some more right now. In the meantime, have you tried adding a minor task to your deploy script that clears the cached files? May be the fastest way.

[thumblemonks]

Looking at it some more, it really seems like rails should be taking advantage of the rails_asset_id for cached files as well. But not in the cached file, on the file bundle instead. Hmmm.

[ari]

You are much more responsive than most.... so don't worry!

I also thought of creating a /public/javascripts/cache folder for these files, but that breaks some javascript which doesn't expect to drop an extra folder deeper (eg. jquery-ui).

I don't actually have a deploy script at all. 'git pull' is all I need to do. Gosh, that thing you found is ancient and completely irrelevant now. But perhaps returning to that sort of idea is probably for the best if there is no way to hack Rails to do the sensible thing in itself.

Rails has this asset id idea, but it seems like a very limited feature only used to get around proxy servers that don't behave correctly.

[thumblemonks]

¡Hola!

I haven't forgotten about this issue. Actually, I've been cheating on Smurf a bit. I've been working on a project called AnimalCracker which I think might be able to replace Smurf entirely as well as work in Rails, Sinatra, any other Rack system. I'll try to put some doc up about it and also include the css/js minification stuff over the weekend.

If this works, we can forget about hacking Rails to fix their silly problem. You'll let me know if this is not suitable :)

[ari]

Not sure how you are going to improve upon the simplicity of Smurf. No configuration, no installation other than adding a gem config line.

[ari]

I really am not sure what AnimalCracker is supposed to do. Could you put some docs up there with the rough outline of what it does?

I've still not given up on getting Smurf to delete the asset files. After all, it can create them when needed. What is involved in triggering it to recreate them every time the server is restarted (depending on the relative modification dates)?

[ari]

Any thoughts about how we can deal with this issue? It is really quite painful and smurf could potentially solve this in a really nice way.

[thumblemonks]

I've really been sitting on this one it seems. The best thought I can come up with at the moment is to add some code in an initializer that globs on public/**/cache/* and deletes those files. It's hard to simply add something to Smurf since Smurf is only really aware of anything when stylesheet or javascript files are referenced in the view (at runtime).

If not in an initializer, it still makes sense to me to add this to the deploy script. In practice of using Capistrano and Vlad the Deployer, I avoid this problem altogether since a new, time-stamped directory is generated on deployment from my project's repo. The effect is that only committed code is deployed, the generated css and js files are left in the previous deployment(s).

[ari]

I find the **/cache directory idea doesn't work for me because I have relative references that can't be moved down a folder in the structure.

I've got a rake script to delete the files now as part of deployment, so I personally have a workaround, but it seems like a bit failing in Rails not to have a mechanism to deal with stale cache files.

What about having Smurf compare the datestamp of the compressed file with that of the source files at runtime. Could it do that no more often than once every 10 minutes during runtime (so it isn't going to disk too often)?