search

skylightio / skylight-ruby

Skylight agent for Ruby
https://www.skylight.io/
Other
314 stars 75 forks source link

Skylight gem causes Heroku deploy to hang under Ruby 3.0.0 #167

Closed ryanfb closed 3 years ago

ryanfb commented 3 years ago

In a Rails 6.1.0 app with the skylight gem and everything else working under Ruby 3.0.0, if I deploy to Heroku the bootup process will log a line like:

I, [2021-01-06T10:19:48.679131 #815]  INFO -- : [SKYLIGHT] [5.0.0-beta4] Skylight agent enabled

…and then it hangs forever. If I remove the skylight gem and redeploy, the app boots without issue. The same app deploys and boots fine under Ruby 2.7.2 with skylight enabled.

(Note: this also happens with v4.3.2 of the gem, I just also tried git master to be sure)

wagenet commented 3 years ago

@ryanfb my apologies for not replying to this sooner! Can you set SKYLIGHT_ENABLE_TRACE_LOGS=true in the environment and see if more details are visible?

ryanfb commented 3 years ago

Unfortunately, that doesn't seem to log any additional lines for me before/after the Skylight agent enabled line before hanging.

wagenet commented 3 years ago

Ok, thanks for checking. One of my colleagues will be looking into this issue. Thanks for your patience!

zvkemp commented 3 years ago

@ryanfb so far we haven't been able to reproduce the issue with Ruby 3 on Heroku. It's possible this is due to an interaction with another gem. Are you able to share your Gemfile.lock? If you are unable to paste it here you can send it directly to us at support@skylight.io.

Thanks in advance!

ryanfb commented 3 years ago

Sure, no problem. One other thing to note is that the app is on the Heroku-20 stack.

Gemfile.lock:

GIT
  remote: https://github.com/skylightio/skylight-ruby.git
  revision: dff13917c71cab9adceaa2d1bce1a815f379ab67
  specs:
    skylight (5.0.0.beta4)
      activesupport (>= 5.2.0)

GEM
  remote: https://rubygems.org/
  specs:
    actioncable (6.1.1)
      actionpack (= 6.1.1)
      activesupport (= 6.1.1)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
    actionmailbox (6.1.1)
      actionpack (= 6.1.1)
      activejob (= 6.1.1)
      activerecord (= 6.1.1)
      activestorage (= 6.1.1)
      activesupport (= 6.1.1)
      mail (>= 2.7.1)
    actionmailer (6.1.1)
      actionpack (= 6.1.1)
      actionview (= 6.1.1)
      activejob (= 6.1.1)
      activesupport (= 6.1.1)
      mail (~> 2.5, >= 2.5.4)
      rails-dom-testing (~> 2.0)
    actionpack (6.1.1)
      actionview (= 6.1.1)
      activesupport (= 6.1.1)
      rack (~> 2.0, >= 2.0.9)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
    actiontext (6.1.1)
      actionpack (= 6.1.1)
      activerecord (= 6.1.1)
      activestorage (= 6.1.1)
      activesupport (= 6.1.1)
      nokogiri (>= 1.8.5)
    actionview (6.1.1)
      activesupport (= 6.1.1)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.1, >= 1.2.0)
    activejob (6.1.1)
      activesupport (= 6.1.1)
      globalid (>= 0.3.6)
    activemodel (6.1.1)
      activesupport (= 6.1.1)
    activerecord (6.1.1)
      activemodel (= 6.1.1)
      activesupport (= 6.1.1)
    activestorage (6.1.1)
      actionpack (= 6.1.1)
      activejob (= 6.1.1)
      activerecord (= 6.1.1)
      activesupport (= 6.1.1)
      marcel (~> 0.3.1)
      mimemagic (~> 0.3.2)
    activesupport (6.1.1)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
      tzinfo (~> 2.0)
      zeitwerk (~> 2.3)
    acts_as_list (1.0.3)
      activerecord (>= 4.2)
    addressable (2.7.0)
      public_suffix (>= 2.0.2, < 5.0)
    archer-rails (0.2.2)
      activerecord (>= 5)
      railties (>= 5)
    ast (2.4.1)
    bcrypt (3.1.16)
    bindex (0.8.1)
    bootsnap (1.5.1)
      msgpack (~> 1.0)
    bootstrap_form (4.5.0)
      actionpack (>= 5.2)
      activemodel (>= 5.2)
    brakeman (4.10.1)
    brotli (0.2.3)
    builder (3.2.4)
    bullet (6.1.2)
      activesupport (>= 3.0.0)
      uniform_notifier (~> 1.11)
    bundler-audit (0.7.0.1)
      bundler (>= 1.2.0, < 3)
      thor (>= 0.18, < 2)
    byebug (11.1.3)
    cancancan (3.2.1)
    capybara (3.34.0)
      addressable
      mini_mime (>= 0.1.3)
      nokogiri (~> 1.8)
      rack (>= 1.6.0)
      rack-test (>= 0.6.3)
      regexp_parser (~> 1.5)
      xpath (~> 3.2)
    childprocess (3.0.0)
    coderay (1.1.3)
    coffee-rails (5.0.0)
      coffee-script (>= 2.2.0)
      railties (>= 5.2.0)
    coffee-script (2.4.1)
      coffee-script-source
      execjs
    coffee-script-source (1.12.2)
    concurrent-ruby (1.1.7)
    crack (0.4.5)
      rexml
    crass (1.0.6)
    curb (0.9.11)
    daemons (1.3.1)
    dalli (2.7.11)
    delayed_job (4.1.9)
      activesupport (>= 3.0, < 6.2)
    delayed_job_active_record (4.1.5)
      activerecord (>= 3.0, < 6.2)
      delayed_job (>= 3.0, < 5)
    devise (4.7.3)
      bcrypt (~> 3.0)
      orm_adapter (~> 0.1)
      railties (>= 4.1.0)
      responders
      warden (~> 1.2.3)
    domain_name (0.5.20190701)
      unf (>= 0.0.5, < 1.0.0)
    dotenv (2.7.6)
    dotenv-rails (2.7.6)
      dotenv (= 2.7.6)
      railties (>= 3.2)
    erubi (1.10.0)
    execjs (2.7.0)
    faraday (1.1.0)
      multipart-post (>= 1.2, < 3)
      ruby2_keywords
    faraday_middleware (1.0.0)
      faraday (~> 1.0)
    fastimage (2.2.1)
    feedjira (2.2.0)
      faraday (>= 0.9)
      faraday_middleware (>= 0.9)
      loofah (>= 2.0)
      sax-machine (>= 1.0)
    feedjira-podcast (0.10.0)
      addressable (~> 2.3)
      feedjira (~> 2.0)
    ffi (1.14.2)
    font_awesome5_rails (1.3.0)
      railties (>= 4.2)
    friendly_id (5.4.2)
      activerecord (>= 4.0.0)
    git-version-bump (0.17.1)
    globalid (0.4.2)
      activesupport (>= 4.2.0)
    guess_html_encoding (0.0.11)
    hashdiff (1.0.1)
    hashie (4.1.0)
    high_voltage (3.1.2)
    honeybadger (4.7.2)
    htmlentities (4.3.4)
    httmultiparty (0.3.16)
      httparty (>= 0.7.3)
      mimemagic
      multipart-post
    http-accept (1.7.0)
    http-cookie (1.0.3)
      domain_name (~> 0.5)
    httparty (0.18.1)
      mime-types (~> 3.0)
      multi_xml (>= 0.5.2)
    i18n (1.8.7)
      concurrent-ruby (~> 1.0)
    iniparse (1.5.0)
    jbuilder (2.10.1)
      activesupport (>= 5.0.0)
    listen (3.4.0)
      rb-fsevent (~> 0.10, >= 0.10.3)
      rb-inotify (~> 0.9, >= 0.9.10)
    lograge (0.11.2)
      actionpack (>= 4)
      activesupport (>= 4)
      railties (>= 4)
      request_store (~> 1.0)
    loofah (2.8.0)
      crass (~> 1.0.2)
      nokogiri (>= 1.5.9)
    mail (2.7.1)
      mini_mime (>= 0.1.1)
    marcel (0.3.3)
      mimemagic (~> 0.3.2)
    method_source (1.0.0)
    mime-types (3.3.1)
      mime-types-data (~> 3.2015)
    mime-types-data (3.2020.0512)
    mimemagic (0.3.5)
    mini_mime (1.0.2)
    mini_portile2 (2.5.0)
    minitest (5.14.3)
    minitest-ci (3.4.0)
      minitest (>= 5.0.6)
    msgpack (1.3.3)
    multi_xml (0.6.0)
    multipart-post (2.1.1)
    netrc (0.11.0)
    nio4r (2.5.4)
    nokogiri (1.11.1)
      mini_portile2 (~> 2.5.0)
      racc (~> 1.4)
    oj (3.11.0)
    open_uri_redirections (0.2.1)
    orm_adapter (0.5.0)
    overcommit (0.57.0)
      childprocess (>= 0.6.3, < 5)
      iniparse (~> 1.4)
    parallel (1.20.1)
    parser (3.0.0.0)
      ast (~> 2.4.1)
    pg (1.2.3)
    postrank-uri (1.0.24)
      addressable (>= 2.4.0)
      nokogiri (>= 1.8.0)
      public_suffix (>= 2.0.0, < 2.1)
    pry (0.13.1)
      coderay (~> 1.1)
      method_source (~> 1.0)
    pry-rails (0.3.9)
      pry (>= 0.10.4)
    public_suffix (2.0.5)
    puma (5.1.1)
      nio4r (~> 2.0)
    racc (1.5.2)
    rack (2.2.3)
    rack-attack (6.3.1)
      rack (>= 1.0, < 3)
    rack-brotli (1.0.0)
      brotli (>= 0.1.7)
      git-version-bump (>= 0.15)
      rack (>= 1.4)
    rack-canonical-host (1.0.0)
      addressable (> 0, < 3)
      rack (>= 1.0.0, < 3)
    rack-cors (1.1.1)
      rack (>= 2.0.0)
    rack-proxy (0.6.5)
      rack
    rack-test (1.1.0)
      rack (>= 1.0, < 3)
    rails (6.1.1)
      actioncable (= 6.1.1)
      actionmailbox (= 6.1.1)
      actionmailer (= 6.1.1)
      actionpack (= 6.1.1)
      actiontext (= 6.1.1)
      actionview (= 6.1.1)
      activejob (= 6.1.1)
      activemodel (= 6.1.1)
      activerecord (= 6.1.1)
      activestorage (= 6.1.1)
      activesupport (= 6.1.1)
      bundler (>= 1.15.0)
      railties (= 6.1.1)
      sprockets-rails (>= 2.0.0)
    rails-dom-testing (2.0.3)
      activesupport (>= 4.2.0)
      nokogiri (>= 1.6)
    rails-html-sanitizer (1.3.0)
      loofah (~> 2.3)
    railties (6.1.1)
      actionpack (= 6.1.1)
      activesupport (= 6.1.1)
      method_source
      rake (>= 0.8.7)
      thor (~> 1.0)
    rainbow (3.0.0)
    rake (13.0.3)
    rb-fsevent (0.10.4)
    rb-inotify (0.10.1)
      ffi (~> 1.0)
    regexp_parser (1.8.2)
    request_store (1.5.0)
      rack (>= 1.4)
    responders (3.0.1)
      actionpack (>= 5.0)
      railties (>= 5.0)
    rest-client (2.1.0)
      http-accept (>= 1.7.0, < 2.0)
      http-cookie (>= 1.0.2, < 2.0)
      mime-types (>= 1.16, < 4.0)
      netrc (~> 0.8)
    rexml (3.2.4)
    rss (0.2.9)
      rexml
    rubocop (1.8.1)
      parallel (~> 1.10)
      parser (>= 3.0.0.0)
      rainbow (>= 2.2.2, < 4.0)
      regexp_parser (>= 1.8, < 3.0)
      rexml
      rubocop-ast (>= 1.2.0, < 2.0)
      ruby-progressbar (~> 1.7)
      unicode-display_width (>= 1.4.0, < 3.0)
    rubocop-ast (1.4.0)
      parser (>= 2.7.1.5)
    rubocop-performance (1.9.2)
      rubocop (>= 0.90.0, < 2.0)
      rubocop-ast (>= 0.4.0)
    rubocop-rails (2.9.1)
      activesupport (>= 4.2.0)
      rack (>= 1.1)
      rubocop (>= 0.90.0, < 2.0)
    ruby-progressbar (1.11.0)
    ruby-readability (0.7.0)
      guess_html_encoding (>= 0.0.4)
      nokogiri (>= 1.6.0)
    ruby2_keywords (0.0.2)
    rubyzip (2.3.0)
    sax-machine (1.3.2)
    selenium-webdriver (3.142.7)
      childprocess (>= 0.5, < 4.0)
      rubyzip (>= 1.2.2)
    semantic_range (2.3.0)
    sitemap (0.3.3)
    soundcloud (0.3.4)
      hashie
      httmultiparty (~> 0.3.0)
    spring (2.1.1)
    spring-watcher-listen (2.0.1)
      listen (>= 2.7, < 4.0)
      spring (>= 1.2, < 3.0)
    sprockets (4.0.2)
      concurrent-ruby (~> 1.0)
      rack (> 1, < 3)
    sprockets-rails (3.2.2)
      actionpack (>= 4.0)
      activesupport (>= 4.0)
      sprockets (>= 3.0.0)
    sqlite3 (1.4.2)
    thor (1.0.1)
    turbo-rails (0.5.4)
      rails (>= 6.0.0)
    tzinfo (2.0.4)
      concurrent-ruby (~> 1.0)
    unf (0.1.4)
      unf_ext
    unf_ext (0.0.7.7)
    unicode-display_width (2.0.0)
    uniform_notifier (1.13.0)
    vcr (6.0.0)
    warden (1.2.9)
      rack (>= 2.0.9)
    web-console (4.1.0)
      actionview (>= 6.0.0)
      activemodel (>= 6.0.0)
      bindex (>= 0.4.0)
      railties (>= 6.0.0)
    webdrivers (4.4.2)
      nokogiri (~> 1.6)
      rubyzip (>= 1.3.0)
      selenium-webdriver (>= 3.0, < 4.0)
    webmock (3.11.1)
      addressable (>= 2.3.6)
      crack (>= 0.3.2)
      hashdiff (>= 0.4.0, < 2.0.0)
    webpacker (5.2.1)
      activesupport (>= 5.2)
      rack-proxy (>= 0.6.1)
      railties (>= 5.2)
      semantic_range (>= 2.3.0)
    websocket-driver (0.7.3)
      websocket-extensions (>= 0.1.0)
    websocket-extensions (0.1.5)
    xpath (3.2.0)
      nokogiri (~> 1.8)
    yard (0.9.26)
    zeitwerk (2.4.2)

PLATFORMS
  ruby

DEPENDENCIES
  acts_as_list
  addressable (~> 2.7)
  archer-rails
  bootsnap (>= 1.1.0)
  bootstrap_form (~> 4.5)
  brakeman (~> 4.9)
  bullet
  bundler-audit (~> 0.7.0)
  byebug
  cancancan (~> 3.1)
  capybara (>= 2.15)
  coffee-rails (~> 5)
  curb
  daemons (~> 1.3)
  dalli
  delayed_job (~> 4.1)
  delayed_job_active_record (~> 4.1)
  devise
  dotenv-rails
  fastimage (~> 2.2)
  feedjira (~> 2.0)
  feedjira-podcast
  font_awesome5_rails
  friendly_id (~> 5.4.0)
  high_voltage (~> 3.1)
  honeybadger (~> 4.0)
  htmlentities (~> 4.3)
  jbuilder (~> 2.5)
  listen (~> 3.0)
  lograge (~> 0.11.2)
  minitest-ci
  nokogiri (>= 1.11.0.rc4)
  oj (~> 3.10)
  open_uri_redirections
  overcommit (~> 0.57.0)
  pg (~> 1.1)
  postrank-uri
  pry-rails
  puma (< 6)
  rack-attack (~> 6.3)
  rack-brotli (~> 1.0)
  rack-canonical-host
  rack-cors
  rails (~> 6.1.0)
  rest-client
  rss
  rubocop (~> 1.0)
  rubocop-performance
  rubocop-rails (~> 2.8)
  ruby-readability
  selenium-webdriver
  sitemap
  skylight!
  soundcloud
  spring
  spring-watcher-listen (~> 2.0.0)
  sqlite3 (~> 1.4, < 1.5)
  turbo-rails (~> 0.5)
  tzinfo-data
  vcr (~> 6.0)
  web-console (>= 3.3.0)
  webdrivers (~> 4.0)
  webmock
  webpacker
  yard

RUBY VERSION
   ruby 3.0.0p0

BUNDLED WITH
   2.2.4

Corresponding Gemfile:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '3.0.0'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 6.1.0'

# Use Puma as the app server
gem 'puma', '< 6'
# Use SCSS for stylesheets
# Use Webpacker as compressor for JavaScript assets
gem 'webpacker'
# See https://github.com/rails/execjs#readme for more supported runtimes
# gem 'mini_racer', platforms: :ruby
# Use CoffeeScript for .coffee assets and views
gem 'coffee-rails', '~> 5'
# https://turbo.hotwire.dev/
gem 'turbo-rails', '~> 0.5'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.5'
# Use Redis adapter to run Action Cable in production
# gem 'redis', '~> 4.0'
# Use ActiveModel has_secure_password
# gem 'bcrypt', '~> 3.1.7'

# Use ActiveStorage variant
# gem 'mini_magick', '~> 4.8'

# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.1.0', require: false

gem 'acts_as_list'
gem 'high_voltage', '~> 3.1'

group :production do
  gem 'pg', '~> 1.1'
  # gem 'heroku-deflater', git: 'https://github.com/romanbsd/heroku-deflater.git', ref: '5868309'
  gem 'archer-rails'
  gem 'dalli'
  gem 'rack-canonical-host'
end

group :development, :test do
  # Use sqlite3 as the database for Active Record
  gem 'sqlite3', '~> 1.4', '< 1.5'
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: %i[mri mingw x64_mingw]
end

group :development do
  # Access an interactive console on exception pages or by calling 'console' anywhere in the code.
  gem 'listen', '~> 3.0'
  gem 'web-console', '>= 3.3.0'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'bullet'
  gem 'pry-rails'
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
end

group :test do
  # Adds support for Capybara system testing and selenium driver
  gem 'capybara', '>= 2.15'
  gem 'selenium-webdriver'
  # Easy installation and use of chromedriver to run system tests with Chrome
  # gem 'chromedriver-helper'
  gem 'vcr', '~> 6.0'
  gem 'webdrivers', '~> 4.0'
  gem 'webmock'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: %i[mingw mswin x64_mingw jruby]

# gem "airbrake", "~> 11.0"
gem 'addressable', '~> 2.7'
gem 'bootstrap_form', '~> 4.5'
gem 'brakeman', '~> 4.9'
gem 'bundler-audit', '~> 0.7.0'
gem 'cancancan', '~> 3.1'
gem 'curb'
gem 'daemons', '~> 1.3'
gem 'delayed_job', '~> 4.1'
gem 'delayed_job_active_record', '~> 4.1'
gem 'devise'
gem 'dotenv-rails', groups: %i[development test]
gem 'fastimage', '~> 2.2'
gem 'feedjira', '~> 2.0'
gem 'feedjira-podcast'
gem 'font_awesome5_rails'
gem 'friendly_id', '~> 5.4.0'
gem 'honeybadger', '~> 4.0'
gem 'htmlentities', '~> 4.3'
gem 'lograge', '~> 0.11.2'
gem 'minitest-ci'
gem 'nokogiri', '>= 1.11.0.rc4'
gem 'oj', '~> 3.10'
gem 'open_uri_redirections'
gem 'overcommit', '~> 0.57.0'
gem 'postrank-uri'
gem 'rack-attack', '~> 6.3'
gem 'rack-brotli', '~> 1.0', require: 'rack/brotli'
gem 'rack-cors', require: 'rack/cors'
gem 'rest-client'
gem 'rss'
gem 'rubocop', '~> 1.0', require: false
gem 'rubocop-performance', require: false
gem 'rubocop-rails', '~> 2.8', require: false
gem 'ruby-readability', require: 'readability'
gem 'sitemap'
gem 'skylight', git: 'https://github.com/skylightio/skylight-ruby.git'
gem 'soundcloud'
gem 'yard'
zvkemp commented 3 years ago

@ryanfb we are able to deploy a test app to heroku-20 using this gemfile. The only thing I see that may be of concern is airbrake, for which certain versions conflict with Skylight (due to a cultural shift away from alias method chaining in favor of Module#prepend; unfortunately mixing the two strategies in the same VM can cause an infinite recursion). I see it's commented out here, but was airbrake active when you deployed with Skylight? Do you have any other internal code that does an alias method chain on e.g., Net::HTTP?

ryanfb commented 3 years ago

airbrake wasn't active when I was encountering the issue (using honeybadger now instead). I've tried without honeybadger as well, but still get the same issue with hanging unless I remove skylight. I'm not using alias method chaining anywhere else, so I'm not entirely sure what's happening. I might try to poke around more and see if I can get any more diagnostic log output during bootup to see if it's possible to more accurately pinpoint where it's hanging.

ryanfb commented 3 years ago

Closing. Not sure what combination of stack/gem changes made it happen, but I just tried a Heroku deploy with the skylight gem on Ruby 3.0.1 and the app boots fine!