search

sass / sassc-ruby

Use libsass with Ruby!
MIT License
367 stars 156 forks source link

Segmentation fault in sassc-2.2.1 when I visit ActiveAdmin in production #167

Open ndbroadbent opened 4 years ago

ndbroadbent commented 4 years ago

Hello,

I upgraded sassc to 2.2.1, and my production Rails app is now crashing when I visit my ActiveAdmin dashboard. Here are the crash logs from my server: https://gist.github.com/ndbroadbent/3d0f78235dc9056ca9eb9464d53b4a9c

/app/vendor/bundle/ruby/2.5.0/gems/sassc-2.2.1/lib/sassc/engine.rb:42: [BUG] Segmentation fault at 0x0000000000000000

Versions:

I'm not too sure why ActiveAdmin is calling SassC in production, but I think it might be part of a third-party template that needs to be rendered at runtime.

Another thing to note is that this consistently crashes my app on every request.

Thanks for your help!

Potentially related issues:

ndbroadbent commented 4 years ago

Sorry, this is also happening with sassc 2.0.1. I upgraded a few gem dependencies at the same time (including ActiveAdmin), so I'm not sure what started causing this crash, but it does seem to be coming from sassc or libsass.

ahorek commented 4 years ago

I reported the same problem a year ago https://github.com/rails/sprockets/issues/581 and I propose a fix, but there's still no response https://github.com/sass/sassc-ruby/pull/162 and https://github.com/rails/sprockets/pull/630

the second option is disabling concurrency, it could still break in some cases, but it'll be much less likely

Rails.application.config.assets.configure do |env|
  env.export_concurrent = false
end
ndbroadbent commented 4 years ago

Thanks for the suggestion @ahorek! Unfortunately I don't think that export_concurrent setting would work for me, because this is not related to sprockets in my case. It's not happening in the assets:precompile task, it's actually happening when I make a request during production, because ActiveAdmin renders some templates (and SCSS) at runtime.

ahorek commented 4 years ago

hmm, do you really need dynamic assets compiled in production? in your case sassc is acually called from tilt https://github.com/rtomayko/tilt/blob/master/lib/tilt/sass.rb#L28 maybe one of your gems is mutating sassc functions ( https://github.com/sass/sassc-ruby/blob/master/lib/sassc/script/functions.rb ) at runtime the same way as sprockets do?

it would be helpful if you have a repro script...

ndbroadbent commented 4 years ago

hmm, do you really need dynamic assets compiled in production?

Yes I'm not sure why ActiveAdmin needs to compile sass in production (this is not coming from my own code or assets.) I would like to know if there's a way to disable it, or maybe precompile these assets. But this seems to be the default behavior in the latest version of ActiveAdmin.

I think it might be difficult to consistently reproduce this crash, but I could try to put together a Docker image and test Rails app.

ndbroadbent commented 4 years ago

Ahhhhhh this is happening because I was lazy and added some inline SCSS in my HAML template:

:scss
  .filters {
    margin-top: 18px;
    margin-bottom: 10px;
    margin-left: 39px;

    .button {
      font-size: 12px;
    }
  }

At least this explains why my ActiveAdmin view is running sassc. So a workaround is to just move this style into my precompiled assets, which should solve the problem for now.

ndbroadbent commented 4 years ago

I've tracked down the cause of the issue. This started happening after I updated the sqreen Ruby gem (https://www.sqreen.com). This gem adds some instrumentation and wraps some methods in Rails, so their code was causing this multi-threading issue that resulted in a segmentation fault (probably a similar issue to sprockets, and other issues that people have reported.)

If anyone wants to look into this, you can see all the middleware and instrumentation lines in the stack trace: sqreen-1.18.1. The sqreen code isn't open source on GitHub, but you can view the source code by running gem install sqreen and then gem open sqreen.

lloeki commented 4 years ago

@ndbroadbent Sqreen Ruby agent engineer here. We tracked down this issue previously to conflicting unmangled libc++ symbols, and fixed it in libsqreen 0.3.0.0.3 (a native dependency of the Sqreen gem). This should be resolved with a bundle update.

BTW we’re working on progressively opening our GitHub repos but we need a git history audit first.

Lorel commented 4 years ago

@ndbroadbent fix confirmed here 🎉 just deployed the version 1.18.2 of the gem sqreen, using libsqreen version 0.3.0.0.3, and I've just succeeded to precompile my assets