search

rails / sprockets

Rack-based asset packaging system
MIT License
949 stars 789 forks source link

Sprockets::FileOutsidePaths error seems to be caused by fix to #59 #96

Closed bughit closed 9 years ago

bughit commented 9 years ago

Not a 100% sure, but what seems to be happening is even though cache key generation now uses relative paths, some or all depedency references are absolute.

so if you are sharing the asset cache folder among multiple release, sprockets ends up finding the right cache entry but then tries to load dependencies from an older release absolute path (where the cache was originally constructed) and fails with Sprockets::FileOutsidePaths

schneems commented 9 years ago

some or all dependency references are absolute.

If you're using an asset that isn't inside of the root of where you're running sprockets, an absolute path will be used to generate the cache key. Can you give me an example project that shows the problem?

bughit commented 9 years ago

I haven't yet tried in a new project but can give you repro directions:

  1. assets:clobber
  2. make a copy of your project_folder (project_copy)
  3. precompile in project_copy to generate the cache
  4. symlink project_folder assets cache to project_copy
  5. if you examine/grep the cache files you'll see absolute references to project_copy (file-digest: urls)
  6. in project_folder there is a javascript_include_tag 'jquery.sparkline'
  7. jquery.sparkline is a folder with an index that //= require ./jquery.sparkline-2.1.2
  8. javascript_include_tag 'jquery.sparkline' raises saying that the absolute path to jquery.sparkline-2.1.2.js via project_copy is no longer under a load path (all the load paths being under project_folder)
bughit commented 9 years ago

so this is actually very easy to repro with a new project

  1. create
  2. add one action that will load application.js via the layout
  3. add one js asset to app/assets/javascripts (application.js already requires tree)
  4. rails server and load the action (it should work)
  5. assets:clobber
  6. make a copy of the project
  7. assets:precompile in the copy
  8. symlink to the copy's cache/assets
  9. rails server and load the action
  10. Sprockets::FileOutsidePaths
bughit commented 9 years ago

with v3.2.0 there's no error, but only because the cache is not used due to absolute paths

with v2.12.4 there's no error and the cache is used

schneems commented 9 years ago

There's a lot of grey area between 1 and 10. I'll attempt trying this tomorrow,

bughit commented 9 years ago

Symlink the tmp/cache in the copy to the original project? Is that what you're doing in a project?

the original project does not have the asset cache (assets:clobber in step 5) the copy has the cache (assets:precompile in step 7) so you need to make the copy's cache available to the original project by creating a symlink to it

schneems commented 9 years ago

I tried that and couldn't reproduce

I'm pointing at my copy's assets and cache:

$ ls -la
total 64
drwxr-xr-x  19 richardschneeman  staff   646 Aug 14 11:56 .
drwxr-xr-x  63 richardschneeman  staff  2142 Aug 14 11:52 ..
drwxr-xr-x  12 richardschneeman  staff   408 Aug 14 11:52 .git
-rw-r--r--   1 richardschneeman  staff   474 Aug 14 11:47 .gitignore
-rw-r--r--   1 richardschneeman  staff  1476 Aug 14 11:47 Gemfile
-rw-r--r--   1 richardschneeman  staff  3851 Aug 14 11:47 Gemfile.lock
-rw-r--r--   1 richardschneeman  staff   478 Aug 14 11:47 README.rdoc
-rw-r--r--   1 richardschneeman  staff   249 Aug 14 11:47 Rakefile
drwxr-xr-x   8 richardschneeman  staff   272 Aug 14 11:47 app
drwxr-xr-x   7 richardschneeman  staff   238 Aug 14 11:47 bin
drwxr-xr-x  11 richardschneeman  staff   374 Aug 14 11:47 config
-rw-r--r--   1 richardschneeman  staff   153 Aug 14 11:47 config.ru
drwxr-xr-x   7 richardschneeman  staff   238 Aug 14 11:50 db
drwxr-xr-x   4 richardschneeman  staff   136 Aug 14 11:47 lib
drwxr-xr-x   5 richardschneeman  staff   170 Aug 14 11:49 log
lrwxr-xr-x   1 richardschneeman  staff    28 Aug 14 11:56 public -> ../sprockets-96-copy/public/
drwxr-xr-x   9 richardschneeman  staff   306 Aug 14 11:47 test
lrwxr-xr-x   1 richardschneeman  staff    25 Aug 14 11:56 tmp -> ../sprockets-96-copy/tmp/

Using sprockets 3.3.0

$ bundle list | grep sprockets
  * sprockets (3.3.0)
  * sprockets-rails (2.3.2)

It works:

$ curl localhost:3000/users/new
<!DOCTYPE html>
<html>
<head>
  <title>Sprockets96</title>
  <link rel="stylesheet" media="all" href="/assets/application-0723cb9a2dd5a514d954f70e0fe0b89f6f9f1ae3a375c182f43b5f2b57e9c869.css" data-turbolinks-track="true" />
  <script src="/assets/application-3698babcae573d7cf113fedbe38437ec3bc009e54bc20ccfe4a69d94b7a17732.js" data-turbolinks-track="true"></script>
  <meta name="csrf-param" content="authenticity_token" />
<meta name="csrf-token" content="SYrsgjE9ckapdN6cxl5h+VZKs6JJtk7dgd/8aCkBAyi8rHRuBA2+hHFBHpU5xTOB7whZ/H6zF87l2UUWpgZ+6Q==" />
</head>
<body>

<h1>New User</h1>

<form class="new_user" id="new_user" action="/users" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="mULGsJ5EH1pa6xTh6LTP9hgvZgP8uiHHj701/onQbpdsZF5cq3TTmILe1OgXL52OoW2MXcu/eNTru4yABtcTVg==" />

  <div class="actions">
    <input type="submit" name="commit" value="Create User" />
  </div>
</form>

<a href="/users">Back</a>

</body>
</html>

If you can repro the problem, can you wrap up the two repos you're using in another git repo and throw em' on github for me?

What are you doing IRL to cause this issue? Do you have multiple rails app deploys that are simlinked to the same asset directory is this only to show a repro case?

bughit commented 9 years ago

Don't link public/assets. The presence of precompiled assets bypasses the cache entirely I believe (right?). Also this is dev mode (on demand compilation is on)

The whole point is to have the original project use the cache generated in the copy. Also, I am linking just the tmp/cache/assets not the whole tmp (this probably does not matter)

In capistrano deploys the assets cache is shared between releases. I am dealing with a project in which on demand compilation is on in prod (that will change but not immediately ). On demand compilation tries to use shared cache entries generated by and pointing to previous releases and produces the error (this is exactly what this test demonstrates).

So please try without public/assets linking, if that does not work, I'll upload.

schneems commented 9 years ago

Still working for me:

$ rm -rf public
$ ls -la
total 56
drwxr-xr-x  19 richardschneeman  staff   646 Aug 14 15:24 .
drwxr-xr-x  63 richardschneeman  staff  2142 Aug 14 11:52 ..
drwxr-xr-x  12 richardschneeman  staff   408 Aug 14 11:52 .git
-rw-r--r--   1 richardschneeman  staff   474 Aug 14 11:47 .gitignore
-rw-r--r--   1 richardschneeman  staff  1476 Aug 14 11:47 Gemfile
-rw-r--r--   1 richardschneeman  staff  3851 Aug 14 11:47 Gemfile.lock
-rw-r--r--   1 richardschneeman  staff   478 Aug 14 11:47 README.rdoc
-rw-r--r--   1 richardschneeman  staff   249 Aug 14 11:47 Rakefile
drwxr-xr-x   8 richardschneeman  staff   272 Aug 14 11:47 app
drwxr-xr-x   7 richardschneeman  staff   238 Aug 14 11:47 bin
drwxr-xr-x  11 richardschneeman  staff   374 Aug 14 11:47 config
-rw-r--r--   1 richardschneeman  staff   153 Aug 14 11:47 config.ru
drwxr-xr-x   7 richardschneeman  staff   238 Aug 14 11:50 db
drwxr-xr-x   3 richardschneeman  staff   102 Aug 14 15:26 lib
drwxr-xr-x   5 richardschneeman  staff   170 Aug 14 11:49 log
drwxr-xr-x   2 richardschneeman  staff    68 Aug 14 15:24 public
drwxr-xr-x   9 richardschneeman  staff   306 Aug 14 11:47 test
lrwxr-xr-x   1 richardschneeman  staff    25 Aug 14 11:56 tmp -> ../sprockets-96-copy/tmp/
drwxr-xr-x   3 richardschneeman  staff   102 Aug 14 11:47 vendor
# environments/production.rb
Rails.application.configure do
  # Settings specified here will take precedence over those in config/application.rb.

  # Code is not reloaded between requests.
  config.cache_classes = true

  # Eager load code on boot. This eager loads most of Rails and
  # your application in memory, allowing both threaded web servers
  # and those relying on copy on write to perform better.
  # Rake tasks automatically ignore this option for performance.
  config.eager_load = true

  # Full error reports are disabled and caching is turned on.
  config.consider_all_requests_local       = false
  config.action_controller.perform_caching = true

  # Enable Rack::Cache to put a simple HTTP cache in front of your application
  # Add `rack-cache` to your Gemfile before enabling this.
  # For large-scale production use, consider using a caching reverse proxy like
  # NGINX, varnish or squid.
  # config.action_dispatch.rack_cache = true

  # Disable serving static files from the `/public` folder by default since
  # Apache or NGINX already handles this.
  config.serve_static_files = ENV['RAILS_SERVE_STATIC_FILES'].present?

  # Compress JavaScripts and CSS.
  config.assets.js_compressor = :uglifier
  # config.assets.css_compressor = :sass

  # Do not fallback to assets pipeline if a precompiled asset is missed.
  config.assets.compile = true

  # Asset digests allow you to set far-future HTTP expiration dates on all assets,
  # yet still be able to expire them through the digest params.
  config.assets.digest = true

  # `config.assets.precompile` and `config.assets.version` have moved to config/initializers/assets.rb

  # Specifies the header that your server uses for sending files.
  # config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache
  # config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for NGINX

  # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
  # config.force_ssl = true

  # Use the lowest log level to ensure availability of diagnostic information
  # when problems arise.
  config.log_level = :debug

  # Prepend all log lines with the following tags.
  # config.log_tags = [ :subdomain, :uuid ]

  # Use a different logger for distributed setups.
  # config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

  # Use a different cache store in production.
  # config.cache_store = :mem_cache_store

  # Enable serving of images, stylesheets, and JavaScripts from an asset server.
  # config.action_controller.asset_host = 'http://assets.example.com'

  # Ignore bad email addresses and do not raise email delivery errors.
  # Set this to true and configure the email server for immediate delivery to raise delivery errors.
  # config.action_mailer.raise_delivery_errors = false

  # Enable locale fallbacks for I18n (makes lookups for any locale fall back to
  # the I18n.default_locale when a translation cannot be found).
  config.i18n.fallbacks = true

  # Send deprecation notices to registered listeners.
  config.active_support.deprecation = :notify

  # Use default logging formatter so that PID and timestamp are not suppressed.
  config.log_formatter = ::Logger::Formatter.new

  # Do not dump schema after migrations.
  config.active_record.dump_schema_after_migration = false
end

Started with

$ env SECRET_KEY_BASE=foo RAILS_ENV=production RAILS_SERVE_STATIC_FILES=1 rails s

Result

$ curl http://localhost:3000/assets/application-3698babcae573d7cf113fedbe38437ec3bc009e54bc20ccfe4a69d94b7a17732.js -I
HTTP/1.1 405 Method Not Allowed
Content-Type: text/plain
Content-Length: 18
Cache-Control: no-cache
X-Request-Id: 8b5bdc1e-d662-408f-a752-bb2c1395f327
X-Runtime: 0.000544
Server: WEBrick/1.3.1 (Ruby/2.2.2/2015-04-13)
Date: Fri, 14 Aug 2015 20:29:33 GMT
Connection: Keep-Alive
schneems commented 9 years ago

The presence of precompiled assets bypasses the cache entirely I believe (right?)

Not quite. First sprockets tries to see if there is a valid asset by first using mtime with the cache to make sure the source file in app/assets hasn't changed. If it hasn't and the cache returns a value then another cache lookup will be performed that stores all the "dependencies" of that asset. Each dependency is pulled out and "resolved" (i.e. they check the mtime) to make sure none of them has changed. If they're all still the same then it uses a digest of the dependencies to create another cache key, this one is finally used to pull the asset from the cache. Only then will sprockets check the disk to see if the asset with digest is already in public/assets. I'm not 100% sure that's everything, but it's my understanding of how sprockets work, though it wouldn't be the first time i've been surprised by sprockets.

on demand compilation is on

This is a huge performance penalty to your site, I hope you've got some really good CDN in front of your site so each asset only has to be compiled once, otherwise you're in for a world of pain. I would highly recommend against leaving this option on, I would be curious hearing why it's needed but perhaps in a different conversation to keep this thread focused.

problem

For your use case, can you add the physical location of the assets to sprockets load paths? Sprockets can use multiple paths, though i'm not sure how to set this through rails, that might alleviate the issue. Another option could be symlinking the public/assets directory so that you're pointing the cache and the assets directory to the same project/path-structure.

bughit commented 9 years ago

you are doing a HEAD on application.js and it's responding with method not allowed. You are not hitting the asset cache with that.

I tested doing just a GET on application.js and it works. So, as I directed, you need to invoke an action that does a javascript_include_tag or css.

I'll upload, unless you can retry quickly.

schneems commented 9 years ago

That was the wrong paste. It renders correctly when I visit the page and when I curl without -I

bughit commented 9 years ago

created a repo

Please follow these directions exactly:

  1. use ruby 2.2.2
  2. git clone git@github.com:bughit/asset_cache_repo.git
  3. cd asset_cache_repo/asset_cache_test_copy
  4. bin/rake assets:precompile
  5. cd ../asset_cache_test
  6. bin/rails server -p 3002
  7. load or curl http://localhost:3002/
bughit commented 9 years ago

can you add the physical location of the assets to sprockets load paths?

As I mentioned, sprockets 2 works fine, so that's the workaround we are using. sprockets 2 cache does not use absolute paths for it's keys or anything else (I think there is nothing else in 2.x), which facilitates cache sharing between releases. Sprockets 3 should not be using absolute paths at all either. 3.3 started along this path, but didn't finish.

elia commented 9 years ago

Got the same error in production this morning, the way we do the deploy is by compiling locally and checking the manifest into the repo and uploading to cloudfront (via asset_sync). Here's the backtrace in case it's useful:

Error message   Sprockets::FileOutsidePaths: /home/ubuntu/apps/histreet/releases/20150814095620/app/assets/javascripts/spree/turbolinks.js.coffee is no longer under a load path: 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-0.8.0/opal, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-0.8.0/stdlib, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-0.8.0/lib, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/bundler/gems/opal-jquery-5f6662498a81/lib, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-activesupport-0.1.0/opal, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-rspec-0.4.3/opal, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/gems/opal-rspec-0.4.3/vendor_lib, 
/home/ubuntu/apps/histreet/shared/bundle/ruby/2.2.0/bundler/gems/opal-haml-3996e34c67bb/opal, 
/home/ubuntu/apps/histreet/releases/20150814102012/app/assets/images, 
/home/ubuntu/apps/histreet/releases/20150814102012/app/assets/javascripts, 
/home/ubuntu/apps/histreet/releases/20150814102012/app/assets/stylesheets, 
/home/ubuntu/apps/histreet/releases/20150814102012/vendor/assets/javascripts, 
/home/ubuntu/apps/histreet/releases/20150814102012/vendor/assets/stylesheets, 
[cut]

Looks like relative paths are being confused by the changed root (from 20150814095620 to 20150814102012) which is stored in the cache for some reason.

Also I tried the steps above and I can reproduce.

clemensg commented 9 years ago

I hit the same error when putting some images in vendor/assets/images, deploying with capistrano and compiling the assets on the server. Then, the following error occurs:

Sprockets::FileOutsidePaths:
/var/rails/releases/20150814213520/vendor/assets/images/fancybox_sprite.png is no longer under a load path:
/var/rails/releases/20150814225400/app/assets/images,
/var/rails/releases/20150814225400/app/assets/javascripts,
/var/rails/releases/20150814225400/app/assets/stylesheets,
/var/rails/releases/20150814225400/lib/assets/javascripts,
/var/rails/releases/20150814225400/lib/assets/stylesheets,
/var/rails/releases/20150814225400/vendor/assets/images
...

That fancybox_sprite.png file however, does exist on the server, so it looks like the same sprockets caching bug.

schneems commented 9 years ago

Thanks for the report, I got the example @bughit sent over to reproduce the problem.

Sprockets 3 should not be using absolute paths at all either. 3.3 started along this path, but didn't finish.

Right now if an asset is not inside of your root, then we cache the absolute path.

https://github.com/rails/sprockets/blob/bae6aa70ac6bfc8887ad4d1c814881d24fac5c09/lib/sprockets/loader.rb#L154-L161

My original assumption was all your assets would either be relative to your project root or at a static path such as /Users/richardschneeman/.gem/ruby/2.2.2/gems/twitter-bootstrap-rails-3.2.0/ but that doesn't look to be valid.

I'm not sure what other alternatives we have. If we cache the relative path to the asset ../../../../../../.gem/ruby/2.2.2/gems/twitter-bootstrap-rails-3.2.0/ then this would change when you moved or down a level. I don't know how sprockets 2 caching worked, and I don't know why the caching scheme was changed to use absolute paths. I can dig into this deeper on monday.

bughit commented 9 years ago

Right now if an asset is not inside of your root, then we cache the absolute path.

I don't understand what you are saying there. There are only two assets, application.js and application.css and both are inside the project (app/assets). I am assuming that's what you mean by "inside the root".

The cache is generated in the asset_cache_test_copy by precompile. So you appear to be saying there should not be any absolute paths in the cache files, but clearly there are.

schneems commented 9 years ago

Here's the issue. When we try to load the asset via https://github.com/rails/sprockets/blob/de057b4f13deb2bb5e8684cd06b9b687f5d09792/lib/sprockets/loader.rb#L201-L202 we find a cache entry exists, and bingo:

{:uri=>
  "file:///Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/application.css?type=text/css&id=362558b6fff2447f73fc9fc4e37a7315119a920a68da221e0c7a8a1b0246e723",
 :load_path=>
  "/Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets",
 :filename=>
  "/Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/application.css",
 :name=>"application",
 :logical_path=>"application.css",
 :content_type=>"text/css",
 :source=>
  "/*\n * This is a manifest file that'll be compiled into application.css, which will include all the files\n * listed below.\n *\n * Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets,\n * or any plugin's vendor/assets/stylesheets directory can be referenced here using a relative path.\n *\n * You're free to add application-wide styles to this file and they'll appear at the bottom of the\n * compiled file so the styles you add here take precedence over styles defined in any styles\n * defined in the other CSS/SCSS files in this directory. It is generally better to create a new\n * file per style scope.\n *\n\n\n */\n\n",
 :metadata=>
  {:dependencies=>
    #<Set: {"environment-version",
     "environment-paths",
     "processors:type=text/css&file_type=text/css",
     "file-digest:///Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/application.css",
     "processors:type=text/css&file_type=text/css&pipeline=self",
     "file-digest:///Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets"}>,
   :sass_dependencies=>#<Set: {}>,
   :links=>#<Set: {}>,
   :included=>
    ["file:///Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/application.css?type=text/css&pipeline=self&id=74e025a9f030d490798b9eb416d5a4451a77bec3e06bdbd0b30a45fd1c455629"],
   :charset=>"utf-8",
   :digest=>
    "\xE8\x0E\x8F#\x18\x04>\x8A\xF9M\xDD\xC2\xAD\xADZO\ts\x9A\x8E\xBB2;:\xB3\x1C\xD7\x1DE\xFD\x91\x13",
   :length=>653},
 :dependencies_digest=>
  "\x81X\xC8\xC7$\xAA|\xF7\x11'\xEB\xC4p\xEBI\xF7\x1F^l\x1Ab\xDC!\xAD?\x8ADN\"\x9B\n\xC7",
 :id=>"362558b6fff2447f73fc9fc4e37a7315119a920a68da221e0c7a8a1b0246e723",
 :mtime=>1439675961}

The uri and load_path both have absolute file paths stored in them. The metadata[:dependencies] does as well, but I'm already handling that in the resolve_dependencies method in the same file. It's probably not caught in the test I wrote since the asset already exists on disk in our copy, though i'm not 100% sure why it doesn't try to load the absolute uri to the wrong path https://github.com/rails/sprockets/blob/de057b4f13deb2bb5e8684cd06b9b687f5d09792/test/test_performance.rb#L109-L134.

bughit commented 9 years ago

The uri and load_path both have absolute file paths stored in them. The metadata[:dependencies] does as well

Right, and they shouldn't. There is no good reason to store absolute paths in the cache entry (unless, as you pointed out, the assets is outside the project)

schneems commented 9 years ago

yes

schneems commented 9 years ago

Here's my WIP https://github.com/rails/sprockets/pull/101 needs some major cleanup. Not sure about using presence of a host to share the absolute/relative information but it seems to work fine. This solution works for your test case but is pretty gnarly

dhampik commented 9 years ago

For anyone having that error with vendor/assets (which worked before), I was able to resolve production deploy by adding to config/initializers/assets.rb the following:

# Add additional assets to the asset load path
Rails.application.config.assets.paths << "#{Rails.root}/vendor/assets"

Not sure if it's a correct fix, but it helped me to deploy my app.

schneems commented 9 years ago

101 is now (what I believe) to be a working fix. I'm trying to write a failing test case in 3.x so we don't have to worry about a future regression.

schneems commented 9 years ago

The fix is committed to the 3.x branch and released in https://rubygems.org/gems/sprockets/versions/3.3.2 give it a spin and let me know if you seen any problems

bughit commented 9 years ago

I precompiled a bunch of stuff and I am still seeing absolute paths for relative path requires //= require ./file

bughit commented 9 years ago

My test didn't exercise every possible way to require/load and whatever else sprockets supports, so it's possible things other than require ./file are also still broken

schneems commented 9 years ago

Thanks for the quick reply. Here's all the keys in the metadata hash generated by running the tests:

[:dependencies, :sass_dependencies, :links, :selector_count, :included, :charset, :digest, :length]

We are already handling :dependencies and :included.

I ran tests and it looks like there's two other places where we're storing absolute paths:

@metadata[:links]: #<Set: {"file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/asset/test-b.js?type=application/javascript&id=5a1727512f031078ee607a7af82cbd42e59740e9fae3558614e5b4d0d342362d"}>

and

@metadata[:sass_dependencies]: #<Set: {"/Users/richardschneeman/Documents/projects/sprockets/test/fixtures/source-maps/sass/main.scss"}>

Ugh.

Luckily the pattern of expanding and compressing in the loader.rb should be a quick fix for those two things, however It looks like any processor can add things to your asset's meta hash. I'm not sure if there's other absolute paths that are stored but not covered in the test suite.

The processor shouldn't have to know about expanding/compressing cache paths, but it shouldn't just arbitrarily be able to break your cache with an absolute path. I wish the original author was around for stuff like this, having some kind of a larger picture is easier than playing wackamole.

schneems commented 9 years ago

Stubbed is also a full URI but not tested at all :( https://github.com/rails/sprockets#metadata

bughit commented 9 years ago

the absolute paths that I noticed are in the :required section and are a result of relative requires require ./file, require_tree ., whatever else can do relative

I don't think you covered these in the previous post

schneems commented 9 years ago

Yep, the docs also have a :required key. I'm not able to get :required or :subbed to show up in tests.

Here's my fixture:

In test/fixtures/default:

//= stub interpolation.js

//= require ./gallery.js
//= require noreturn.js
//= require_tree ./vendor/gems/jquery-2-0
//= require_self
//= link "logo.svg"
//= require ./+plus.js

var dog = "cinco";

Here's asset: 

{:uri=>
  "file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/schneems.js.erb?type=application/javascript&id=ebfe3e03d098268cfbeff01680872d26b5a44dc5e9fab54c68efa1016e6ebf70",
 :load_path=>
  "/var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/",
 :filename=>
  "/var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/schneems.js.erb",
 :name=>"schneems",
 :logical_path=>"schneems.js",
 :content_type=>"application/javascript",
 :source=>
  "var Gallery = {};\nvar Foo;\nvar jQuery;\n\n\n\n\n\n\n\n\n\n\nvar dog = \"cinco\";\nfunction plus(a, b) { return a + b; }\n",
 :metadata=>
  {:dependencies=>
    #<Set: {"environment-version",
     "environment-paths",
     "processors:type=application/javascript&file_type=application/javascript&engines=.erb",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/schneems.js.erb",
     "processors:type=application/javascript&file_type=application/javascript&pipeline=self",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/gallery.js",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/noreturn.js",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/vendor/gems/jquery-2-0/jquery.js",
     "processors:type=application/javascript&file_type=application/javascript&engines=.erb&pipeline=self",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/interpolation",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/interpolation.js",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/gallery",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/noreturn",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/vendor/gems/jquery-2-0",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/logo",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/logo.svg",
     "processors:type=image/svg+xml&file_type=image/svg+xml",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/+plus",
     "file-digest:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/+plus.js"}>,
   :links=>
    #<Set: {"file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/logo.svg?type=image/svg+xml&id=8d727e776f3667b8f6e5f80c817cefbf6176ffa56cd59b2c7c70f2796dfd7859"}>,
   :included=>
    ["file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/gallery.js?type=application/javascript&pipeline=self&id=0b2248cdbffcf10edf68b9774a77e7c3de3f74fd8d09daa935ee5882d73cb807",
     "file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/noreturn.js?type=application/javascript&pipeline=self&id=d10461e0c8febe9020353af585d4d15121e83bb72d1edac263bedaf99faa5e78",
     "file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/vendor/gems/jquery-2-0/jquery.js?type=application/javascript&pipeline=self&id=8a3e330d77edea6abab64eccb4f3141983fe5b67b91216efe8b2616292260a24",
     "file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/schneems.js.erb?type=application/javascript&pipeline=self&id=70d86c54c75720e5bdf4a57342ba56540ed448b31b487bd8881680e8b8c812af",
     "file:///var/folders/ss/lf77xfjd1g7ftznjmk3jzw1w0000gn/T/d20150819-86136-1vj0qjq/+plus.js?type=application/javascript&pipeline=self&id=07036c17f6b3aaf8f2cca3f7ee5713cdf33bbd416f3320633faa09d38b6bae9f"],
   :charset=>"utf-8",
   :digest=>
    "u\xCF\xE0\xE9^\xD0(\x87x\xBE\x12\xBB@\xF8;\xC6\xF7\xF3;x\x1E\xEB|G\xD9\xC0\xC8\xEFa\x9BG\xE1",
   :length=>106},
 :dependencies_digest=>
  "h\n\x95\x15!\x10\xA4\x8C\xAD\x19`/d\xAE\xF7\xBE\x90Oa\x8F\xF0\xDB\x8A\x148K\xB9\xD9B4\xB1P",
 :id=>"ebfe3e03d098268cfbeff01680872d26b5a44dc5e9fab54c68efa1016e6ebf70",
 :mtime=>1440007863}
bughit commented 9 years ago

I don't know about the tests, but to see it in vivo is easy.

pull my updated sample, do a precompile in the copy and look at the cache files

schneems commented 9 years ago

Thanks, i'm on it.

schneems commented 9 years ago

This is extremely frustrating. I can clearly see a :required in your example going into the cache

   :required=>
    #<Set: {"file:///Users/richardschneeman/Documents/projects/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/application.css?type=text/css&pipeline=self"}>,
   :stubbed=>#<Set: {}>,
   :links=>#<Set: {}>,

But no matter what I do, I can't get 

env1 = Sprockets::Environment.new(fixture_path('default')) do |env|
  env.append_path(".")
  env.cache = @cache
end

To give me an asset that has a :stubbed or :required metadata key. I can verify that the DirectiveProcessor is being run, which is what looks like it inserts that key. Maybe @guilleiguaran has some idea of what rails might be doing differently than vanilla sprockets?

schneems commented 9 years ago

On the first asset["schneems.js"] call i'm getting two different entries put in the cache:

using

//= stubs "schneems.js"
//= require_self

var dog = "cinco";

I'm seeing two different assets being generated

uri:      "file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self&id=87aebf99a63e757ed0c9b987eaa479110b5a48b8a51867e7f77f413c74e8ae03"
metadata: {:dependencies=>#<Set: {"environment-version", "environment-paths", "processors:type=application/javascript&file_type=application/javascript&pipeline=self", "file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js"}>, :required=>#<Set: {"file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self"}>, :stubbed=>#<Set: {}>, :links=>#<Set: {}>, :charset=>"utf-8", :digest=>"x\xBA\xD9}\xAA\xD4\xDB\xAC\x00Z\x8D\x8A\xF1Y\xA9\x83\x1E\x12\xFC\xA6\x879\x9D\xD5\x930\xEE\xD8F\xF9\bu", :length=>45}
uri:      "file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&id=9d6310e3ec2c2b6b29d0c8dfda084834e6b652263a681837d139c0d07648eeca"
metadata: {:dependencies=>#<Set: {"environment-version", "environment-paths", "processors:type=application/javascript&file_type=application/javascript", "file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js", "processors:type=application/javascript&file_type=application/javascript&pipeline=self"}>, :links=>#<Set: {}>, :included=>["file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self&id=87aebf99a63e757ed0c9b987eaa479110b5a48b8a51867e7f77f413c74e8ae03"], :charset=>"utf-8", :digest=>"x\xBA\xD9}\xAA\xD4\xDB\xAC\x00Z\x8D\x8A\xF1Y\xA9\x83\x1E\x12\xFC\xA6\x879\x9D\xD5\x930\xEE\xD8F\xF9\bu", :length=>45}

One that has required and stubbed, one that doesn't.

rafaelfranca commented 9 years ago

@schneems maybe it is sass-rails and the glob importer?

joker-777 commented 9 years ago

Hi guys,

just wanted to let you know that we are having the same problem.

We make a copy of the old project folder with rsync (including puglic/assets and tmp/cache/assets) and then run precompile on the copied folder. This would result in the following error.

 Sprockets::FileOutsidePaths:
/var/local/kenhub/kenhub.d/111/vendor/bundle/ruby/2.2.0/gems/normalize-rails3.0.3/vendor/assets/stylesheets/normalize-rails.scss is no longer under a load path:
/var/local/kenhub/kenhub.d/113/app/assets/audios,
/var/local/kenhub/kenhub.d/113/app/assets/fonts,
/var/local/kenhub/kenhub.d/113/app/assets/images,
/var/local/kenhub/kenhub.d/113/app/assets/javascripts,
/var/local/kenhub/kenhub.d/113/app/assets/stylesheets,
/var/local/kenhub/kenhub.d/113/vendor/assets/images,
/var/local/kenhub/kenhub.d/113/vendor/assets/javascripts,
/var/local/kenhub/kenhub.d/113/vendor/assets/stylesheets, 
...

as you can see it uses the older folder 111 not the new folder 113.

We upgraded sprockets because the precompilation took really long. We were happy to see that this got fixed here https://github.com/rails/sprockets/issues/59 but obviously introduced this new error.

Thanks for your help!

schneems commented 9 years ago

Caution: brain dump. I'm writing this not for you really, but more for me. If I can't explain the process, I don't really understand it. If you want to know the answer to why we couldn't see those things in the tests, skip to the bottom.

Here's the life cycle of an asset. I reduced the complexity of my javascript to only

var dog = "cinco";

To get things started we use the sprockets api to initiate asset lookup:

env['schneems.js']

It's never been loaded before. We've got the cache on. Sprockets calls resolve on "schneems.js" and resolves the URI to:

"file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript"

This asset isn't in cache so it calls load_from_unloaded in the loader.rb, before it's fully loaded, the we call call_processors on the asset. The first time the asset is passed in the only processor called is

[Sprockets::Bundle]

The "processors" are pulled based on the pipeline query param in the asset uri, if there is none :default will be used:

def processors_for(type, file_type, engine_extnames, pipeline)
  pipeline ||= :default
  config[:pipelines][pipeline.to_sym].call(self, type, file_type, engine_extnames)
end

Processors are defined like:

register_pipeline :default do |env, type, file_type, engine_extnames|
  env.default_processors_for(type, file_type, engine_extnames)
end

When this is called, the file type which is "application/javascript" will be passed in and the "bundle processor" registered for this type will be returned. This happens to be Sprockets::Bundle from:

register_bundle_processor 'application/javascript', Bundle

Now we call this processor on our asset. Bundle#call gets executed with :uri=>"file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript" and :filename=>"/Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js"

We resolve this uri again, after adding an accept, a compat=false and a pipeline=self to the URI. This process also identifies dependencies of the asset. Since this asset is not doing anything else it declares it's own digest as a dependency:

#<Set: {"file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js"}>

It merges this with existing dependencies

#<Set: {"environment-version", "environment-paths", "processors:type=application/javascript&file_type=application/javascript", "file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js"}>

Which produces the same set. It uses the processed_uri which is: "file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self" (I think we're adding the pipeline=self here to avoid infinite recursion as we call load on the URI again.

We then use Utils.dfs to find any "required" files from the processed_uri. This will pull all entries from metadata[:required] of the original asset, and iterate over each of those "required" to find their "required" until it's seen the whole tree.

This was the confusing part to me since we're calling env.load(uri) on the processed_uri which is almost identical to our original URI minus the pipeline=self. This initiates the whole process again i.e. we're calling Environment#load from within Environment#load.

The difference here is that since we're not using the :default "pipeline" we're now using the "self" pipeline which include:

[#<Sprockets::DirectiveProcessor:0x007fc8bc176718 @header_pattern=/\A(?:(?m:\s*)(?:(?:\/\/.*\n?)+|(?:\/\*(?m:.*?)\*\/)|(?:\#.*\n?)+|(?:\#\#\#(?m:.*?)\#\#\#)))+/>, Sprockets::FileReader]

The DirectiveProcessor parses and evaluates directives in a file like:

//= require "foo"

The FileReader processor reads in the input filename, it reads the source of the file into memory, and generates a digest of the content. The data is returned and merged back into the original to finish the "loading" of this asset and a fully resolved Asset object is created (and stored in the cache), it's important to not that the Bundle processor isn't called in this process, otherwise we could never return. Remember that the asset we just generated isn't the one we're looking for, it's an intermediate Asset object, now we rewind our stack all the way back to the Bundle processor where we were when we called this second "load".

We use this second asset which was generated in our Utils.dfs search for all required uris for the original asset loaded.

In this case there is only one "element" in the array. It finds:

required = #<Set: {"file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self"}>

We then do the same thing but start our search with the "stubbed" set from our pipeline=self asset. Since there's no stubbed assets an empty set is returned.

It's important to note that env.load(uri) is memoized, so that multiple calls do not do the whole DirectiveProcessor FileReader dance, only the first call.

Now the "stubbed" and "required" Sets are merged into the original "dependencies" set. The final "dependencies ends up being:

dependencies =#<Set: {"environment-version", "environment-paths", "processors:type=application/javascript&file_type=application/javascript", "file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js", "processors:type=application/javascript&file_type=application/javascript&pipeline=self"}>

Now all of this data is "reduced" the return of this Bundle#call is 

{:data=>"\nvar dog = \"cinco\";\n",
 :links=>#<Set: {}>,
 :dependencies=>
  #<Set: {"environment-version",
   "environment-paths",
   "processors:type=application/javascript&file_type=application/javascript",
   "file-digest:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js",
   "processors:type=application/javascript&file_type=application/javascript&pipeline=self"}>,
 :included=>
  ["file:///Users/richardschneeman/Documents/projects/sprockets/test/fixtures/default/schneems.js?type=application/javascript&pipeline=self&id=7a785b7a0f2f95b15fd0a1b765d9ee974a85991ed94b433e768f597c26393e83"]}

While the second pipeline=self asset has a "required" set and an (empty) "stubbed" set. The primary asset that we are loading does not have these things. Why? Well we don't pass in "stubbed" or "required" to the reducer, so there's no way they could come out. Those two things are intermediate sets used to fully populate the "dependencies" set.

Answer

So stubbed and required are both intermediate objects that are stored in the cache but we CANNOT externally access them without going into private APIs

schneems commented 9 years ago

Here's the patch, https://github.com/rails/sprockets/pull/109 I ended up dipping into a private API for the tests. I think it's better to test this functionality and risk a slightly brittle test than to leave it untested.

schneems commented 9 years ago

Can you try my patch on your own apps, so we hopefully don't have to do another merge/patch/release cycle after this.

You can put this in your gemfile:

gem "sprockets", github: "rails/sprockets", branch: "schneems/fix-metadata-relative-3.x"

We are pending merging and cutting a new sprockets version on your feedback.

cc/ @bughit @joker-777

bughit commented 9 years ago

updated sample, there are 4 cache files with absolute paths

schneems commented 9 years ago

Thanks a ton! I see several absolute paths from gems:

file-digest:///Users/richardschneeman/.gem/ruby/2.2.3/gems/jquery-rails-4.0.4/vendor/assets/javascripts/jquery_ujs.js

This is expected, as they're outside of the root, your gem directory shouldn't be changing between builds that frequently. If people really care about it, they can bundle their into a vendor directory inside of their root. I'm also seeing the absolute path of the source scss file written to the contents of another file:

/* line 1, /Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss */

This is unexpected, I'm not sure where that's being inserted, or what that comment is being used for. I'll have to dig in more.

bughit commented 9 years ago

I wasn't counting the first kind, was greping for project source dir full path, which gave me 4 files, and you?

In addition to the second kind (comment), there was also @file and @root (I think, will double check tomorrow) Are you seeing those?

schneems commented 9 years ago

I can confirm i'm also seeing some file: but these are gems as well

"file:///Users/richardschneeman/.gem/ruby/2.2.3/gems/jquery-rails-4.0.4/vendor/assets/javascripts/jquery.js?type=application/javascript&pipeline=self"

I added a puts to the Cache#set method and here's everything being written to cache along with its key: https://gist.github.com/schneems/1f988a4bcbff8b0c0a3a

It does look like sass writes something else other than the comment:

{:version=>"3.4.16 (Selective Steve)", :sha=>"71853c6197a6a7f222db0f1978c7cb232b87c5ee", :contents=>"\x04\bo:\x19Sass::Tree::RootNode\n:\x0E@children[\x00:\x0E@templateI\"\a\n\n\x06:\x06ET:\n@linei\x06:\x12@source_rangeo:\x18Sass::Source::Range\t:\x0F@start_poso:\eSass::Source::Position\a;\ti\x06:\f@offseti\x06:\r@end_poso;\r\a;\ti\x06;\x0Ei\x06:\n@fileI\"|/Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scss\x06;\bT:\x0E@importero:\x1ESass::Rails::SassImporter\b:\n@rootI\"|/Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scss\x06;\bF:\x0F@real_rootI\"|/Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scss\x06;\bF:\x18@same_name_warningso:\bSet\x06:\n@hash{\x00:\r@options{\x00"}

My root is /Users/richardschneeman/Documents/projects/tmp/asset_cache_repo.

All this sass stuff is new to me. It looks like some (all?) of this is coming from the sass gem directly. If that's the case either Sprockets2 has the same bug, or it ends up not mattering. Not sure without more research.

bughit commented 9 years ago 
grep /tmp/asset_cache_repo/asset_cache_test_copy/ *

Binary file BPk82Q0VktG2GPovwSHPysyr27IyWE5kuwo6FWsTccA.cache matches
Binary file lpNenmFa5RbxzW1angUNN_l6blUaixmPjHrpFoafsgE.cache matches
Binary file PmQLT63FTKauRkinZSc5stYiJXhMun10-pUVsy2bhI0.cache matches
Binary file xfzSrEFVSrlGcqb97NgufRAtPlLchvbikTlsBfVZgaA.cache matches

grep -a /tmp/asset_cache_repo/asset_cache_test_copy/ *

BPk82Q0VktG2GPovwSHPysyr27IyWE5kuwo6FWsTccA.cache:{uriI"file://app/assets/stylesheets/application.css?type=text/css&id=13290f4508text/css;T:p/assets/stylesheets/application.css;T::nameI"application;T:logical_pathI"application.css;T:content_typeI"
             sourceI"#/* line 1, /tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss */
lpNenmFa5RbxzW1angUNN_l6blUaixmPjHrpFoafsgE.cache:@file0:@importer0;i;0;i;i;0:@selector_source_rangeo;    ;o;;i;i;o;;i;i;I"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss;   T;o:Sass::Rails::SassImporte:
lpNenmFa5RbxzW1angUNN_l6blUaixmPjHrpFoafsgE.cache:@rootI"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss;   F:@real_rootI"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss;  F:@same_name_warningso;;{:
PmQLT63FTKauRkinZSc5stYiJXhMun10-pUVsy2bhI0.cache:@fileI"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scssT:@importero:Sass::Rails::SassImporte:
PmQLT63FTKauRkinZSc5stYiJXhMun10-pUVsy2bhI0.cache:@rootI"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scssF:@real_rootI"R/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scssF:@same_name_warningsoSet:
text/css;T:qb97NgufRAtPlLchvbikTlsBfVZgaA.cache:scss2;T:logical_pathI"scss2.self.css;T:content_typeI"
             sourceI"w/* line 1, /tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss2.scss */
schneems commented 9 years ago

The comment is being inserted by https://github.com/sass/sass/blob/1421c1fb38f56d947d1e0271ab5ed12dee510d48/lib/sass/tree/visitors/to_css.rb#L307 and that other absolute paths come from sass cache.

Sass uses marshal dump: https://github.com/sass/sass/blob/stable/lib/sass/cache_stores/base.rb#L51

The nodes it dumps contain a "source_range"

[3] pry(#<Sass::Rails::CacheStore>)> root.inspect
=> "Sass::Tree::RootNode"
# ...
[6] pry(#<Sass::Rails::CacheStore>)> root.source_range
=> (1:1 to 1:1 in /Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/app/assets/stylesheets/scss1.scss)

All the absolute paths are coming from sass.

If there's no absolute paths coming from sprockets (that we can find for now) I vote we merge and cut a release. I'll then work with sass to remove the absolute paths there.

Unfortunately sprockets doesn't have a dependency on sass since you can use sprockets without sass, so we can't rev the sass dependency to guarantee no absolute paths. We could check the version and emit a warning perhaps?

joker-777 commented 9 years ago

I'm testing https://github.com/rails/sprockets/issues/96#issuecomment-13315087 right now and couldn't experience the error again. Looks like as if it is fixed for me.

schneems commented 9 years ago

@joker-777 thanks for testing! I opened an issue with sass https://github.com/sass/sass/issues/1802 hopefully I can get some help there while I work on a patch.

joker-777 commented 9 years ago

@schneems You are welcome. Unfortunately we just discovered that it somehow doesn't recompile js files which were changed. Investigating further.

schneems commented 9 years ago

@joker-777 let me know. I just tested with @bughit's app and it seems to work

2.2.3  ~/documents/projects/tmp/asset_cache_repo/asset_cache_test_copy (master)
$ bin/rake assets:precompile

2.2.3  ~/documents/projects/tmp/asset_cache_repo/asset_cache_test_copy (master)
$ echo 'var foo = "hello";' >> app/assets/javascripts/asset1.js

2.2.3  ~/documents/projects/tmp/asset_cache_repo/asset_cache_test_copy (master)
$ bin/rake assets:precompile
I, [2015-08-21T10:12:08.152279 #66549]  INFO -- : Writing /Users/richardschneeman/Documents/projects/tmp/asset_cache_repo/asset_cache_test_copy/public/assets/application-55ae4c5318c07b2196f18e52480c0563fe6631eb7fd24d38a2e15a5d50a55b59.js