Sprockets::FileOutsidePaths error seems to be caused by fix to #59

[bughit]

Not a 100% sure, but what seems to be happening is even though cache key generation now uses relative paths, some or all depedency references are absolute.

so if you are sharing the asset cache folder among multiple release, sprockets ends up finding the right cache entry but then tries to load dependencies from an older release absolute path (where the cache was originally constructed) and fails with Sprockets::FileOutsidePaths

[joker-777]

Ok, we definitely have this problem that changes js files don't get precompiled. I will clobber the current assets and try it again.

[schneems]

Can you give me an example app that reproduces the problem? I'll be happy to take a look.

[joker-777]

Ok, now it works. We keep and eye on it.

[schneems]

The clobber might have been needed for the cache changes to take affect if the old cache had absolute paths pointing at assets that were still on disk. Thanks again for testing.

[joker-777]

Unfortunately I have tell you that we still experience that js files don't get precompiled even though they should contain changes. In the beginning it worked but now happened again.

[schneems]

Can you give me an example app that reproduces the problem? I'll be happy to take a look.

[JamesChevalier]

I'm chiming in, just in case I can be of any help.

I tried a deploy this morning after updating sprockets, and it failed with a FileOutsidePaths error (details below).

I did get a successful deploy by forcing the version to 3.3.0. Using versions 3.3.2, or 3.3.3 results in the deploy attempting to use the old release path during precompile (below, notice that the css file it's referring to is in 20150821195208 while the load path it's referring to is in 20150824133631).

Sprockets::FileOutsidePaths: /var/www/staging/releases/20150821195208/app/assets/stylesheets/application.css is no longer under a load path: ... /var/www/staging/releases/20150824133631/app/assets/stylesheets ...

The order I went in testing versions of sprockets was:

1. This morning's first deploy after updating to 3.3.3 from 3.3.2 (fail)
2. I rolled back to 3.3.2 (fail)
3. I jumped back to 3.3.0 (success)
4. Now I'm curious which version introduced the issue, so I try 3.3.1 (success)
5. I'm still curious, so I try this again ... 3.3.2 (success)

So for me this doesn't seem to be caused solely by a particular version of sprockets. Whether or not a previous deploy was successful also seems to play a role. Whatever that means. :-)

It seems that version 3.3.0 is the newest version that is capable of completing a deploy, so I'm currently pinned there. Let me know if you'd like more information or have any tests I could run for you.

[schneems]

@JamesChevalier Version 3.3.0 introduced changes to the cache, so you won't see any issue at all unless you are using the cache. Sprockets invalidates the cache on every version change, so every time you upgrade or downgrade sprockets you theoretically should be getting a fresh cache. Since that's the case, you should be getting a problem the second time you deploy, not the first. So deploying with 3.3.0 or 3.3.1 or 3.3.2 might work the first time but probably shouldn't work the second time as there's known bugs in them. If you can give me a 3.3.3 example app that reproduces this error, I would be grateful.

[themilkman]

We had the same issue here and solved it by changing into the old directory used (20150821195208 in your case) and deleting the whole tmp/cache/assets/production dir. Afterwards the deployment worked again - reproducible.

[JamesChevalier]

When using sprockets v 3.3.0, I am able to deploy more than once. I tested simply running cap staging deploy a number of times in a row, with no changes. I also tested by running cap staging deploy, making some minor view code changes, and running cap staging deploy again.

What are the specific files/details you'd like to receive, for the reproduced error? I want to be sure I don't waste your time.

[schneems]

3.3.0 works because works because it has bugs in it, 3.3.3 fixed all known bugs so if you are getting errors with 3.3.3 then we need to reproduce them and fix them. If you can give me a 3.3.3 example app on github that reproduces this error, and instructions on how to run it to reproduce the error then I can probably fix whatever issue you're seeing.

[JamesChevalier]

OK, sounds good.

I've got this app back up to 3.3.3, and it's working fine (something about that 3.3.0 deploy I mentioned above as step 3 in my testing has fixed it for now). If there's any connection between this issue and the fact that there were a few days between deploys, then I don't expect to see any issue with this until Tuesday. Otherwise, it seems like I ran across A Weird Thing in the upgrade & I'm not sure I'll be able to reproduce it at this point.

Thanks for your help & hard work!

[karlingen]

@themilkman @schneems Wonderful! I had the same issue here. Removing tmp/cache/assets/production along with upgrading to 3.3.3 fixed it. Thanks a bunch guys!

[Sija]

@karlingen same here, worked like charm!

[krisdigital]

Upgrading to 3.3 and running

bundle exec rake tmp:clear

on the server before deploying worked.

[schneems]

Awesome, going to close this issue. If you see any other weirdness please open up a new one with information on how to reproduce. Thanks for everyone's help :heart:

[itay-grudev]

@schneems I have no idea what has changed, but I am getting the same issue running 4.0.0.beta6.

[itay-grudev]

I vote for reopening this issue.

[mat2maa]

@schneems @itay-grudev yes, me also - downgrading to 3.7.1 resolves the issue, but I can't do that for the lack of ES6 support. For now, I have a Capistrano before hook which does

bundle exec rake tmp:clear

before starting deployment.

[jjwdowik]

Seeing the same issue as well on 4.0.0.beta6

[marcamillion]

I am seeing this same issue within the test suite on 4.0.2.

I stumbled on another bug which I think I have fixed, but now when I run the test suite, I get this error:

Finished in 140.653251s, 6.3987 runs/s, 28.0050 assertions/s.

  1) Error:
TestLoader#test_"load uri with index alias":
Sprockets::FileOutsidePaths: /Users/marcamillion/sprockets/%2FUsers%2Fmarcgayle%2Fsprockets%2Ftest%2Ffixtures%2Fdefault%2Fcoffee.js (index alias of /Users/marcamillion/sprockets/test/fixtures/default/coffee/index.js) is no longer under a load path: /Users/marcamillion/sprockets/test/fixtures/default
    /Users/marcamillion/sprockets/lib/sprockets/loader.rb:156:in `load_from_unloaded'
    /Users/marcamillion/sprockets/lib/sprockets/loader.rb:59:in `block in load'
    /Users/marcamillion/sprockets/lib/sprockets/loader.rb:337:in `fetch_asset_from_dependency_cache'
    /Users/marcamillion/sprockets/lib/sprockets/loader.rb:43:in `load'
    /Users/marcamillion/sprockets/lib/sprockets/cached_environment.rb:44:in `load'
    /Users/marcamillion/sprockets/lib/sprockets/environment.rb:43:in `load'
    /Users/marcamillion/sprockets/test/test_loader.rb:55:in `block in <class:TestLoader>'

900 runs, 3939 assertions, 0 failures, 1 errors, 4 skips

I am still trying to figure out how to tackle this properly, but I don't fully understand the issue yet.

Edit 1

Hey @schneems could you help me understanding the flow of some of this code and how this works.

I think I have isolated the problem to these lines.

        path_to_split =
          if index_alias = unloaded.params[:index_alias]
            expand_from_root index_alias
          else
            unloaded.filename
          end

        load_path, logical_path = paths_split(config[:paths], path_to_split)

        unless load_path
          target = path_to_split
          target += " (index alias of #{unloaded.filename})" if unloaded.params[:index_alias]
          raise FileOutsidePaths, "#{target} is no longer under a load path: #{self.paths.join(', ')}"
        end

Here are the first 3 variables in action - index_alias, unloaded.params[:index_alias] & path_to_split:

[15] pry(#<Sprockets::CachedEnvironment>)> index_alias
=> "%2FUsers%2Fmarcamillion%2Fsprockets%2Ftest%2Ffixtures%2Fdefault%2Fcoffee.js"
[16] pry(#<Sprockets::CachedEnvironment>)> unloaded.params[:index_alias]
=> "%2FUsers%2Fmarcamillion%2Fsprockets%2Ftest%2Ffixtures%2Fdefault%2Fcoffee.js"
[17] pry(#<Sprockets::CachedEnvironment>)> path_to_split
=> "/Users/marcamillion/sprockets/%2FUsers%2Fmarcamillion%2Fsprockets%2Ftest%2Ffixtures%2Fdefault%2Fcoffee.js"

Then if we look at the 2nd 3 variables:

[18] pry(#<Sprockets::CachedEnvironment>)> config[:paths]
=> ["/Users/marcamillion/sprockets/test/fixtures/default"]
[19] pry(#<Sprockets::CachedEnvironment>)> path_to_split
=> "/Users/marcamillion/sprockets/%2FUsers%2Fmarcamillion%2Fsprockets%2Ftest%2Ffixtures%2Fdefault%2Fcoffee.js"
[20] pry(#<Sprockets::CachedEnvironment>)> paths_split(config[:paths], path_to_split)
=> nil

That nil from paths_split(config[:paths], path_to_split) is what causes this error in my instance I believe.

But I can't quite grok what's happening here and why it creates that funky URI in the path_to_split.

Thoughts?