search

sparklemotion / nokogiri

Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.
https://nokogiri.org/
MIT License
6.13k stars 897 forks source link

Ship Ruby 3.1 precompiled native support #2397

Closed flavorjones closed 2 years ago

flavorjones commented 2 years ago

This is a placeholder to ship a new version of Nokogiri with Ruby 3.1 precompiled native support.

Please note that Nokogiri supports Ruby 3.1, but until a precompiled native gem is shipped, you will have to use the ruby vanilla platform gem which will compile from source during installation. Modern bundler will do this for you automatically.

This work is currently waiting for a version of rake-compiler-dock with Ruby 3.1 build capabilities. The maintainers of that project are aware, and there is no need to bother them. Feel free to subscribe to this issue and I'll keep it updated.

See #2408 for status.

connorshea commented 2 years ago

Not urgent at all, obviously, since it's Christmas :)

Just wanted to note that I ran into this when trying to upgrade a project to Ruby 3.1 and running bundle install:

nokogiri-1.12.5-x86_64-darwin requires ruby version < 3.1.dev, >= 2.5, which is incompatible with the current version, ruby 3.1.0p0
% ruby -v
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin21]

The platforms in my Gemfile.lock include Ruby, so I'm not sure why Bundler isn't falling back to it automatically like (based on the description of this issue?) it should:

PLATFORMS
  ruby
  x86_64-darwin-18
  x86_64-darwin-19
  x86_64-darwin-20
  x86_64-linux

The following is my Gemfile and Gemfile.lock:

Gemfile ``` source 'https://rubygems.org' git_source(:github) { |repo| "https://github.com/#{repo}.git" } ruby '>= 2.7' # Bundle edge Rails instead: gem 'rails', github: 'rails/rails' gem 'rails', '~> 6.1.4' # Use postgresql as the database for Active Record gem 'pg', '~> 1.2' # Use Puma as the app server gem 'puma', '~> 5.5' # Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker gem 'webpacker', '~> 5.4.0' # Turbolinks makes navigating your web application faster. Read more: https://github.com/turbolinks/turbolinks gem 'turbolinks', '~> 5' # Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder gem 'jbuilder', '~> 2.11' # Use devise for Users and authentication. gem 'devise', '~> 4.8' # Use kaminari for pagination. gem 'kaminari', '~> 1.2' # Reduces boot times through caching; required in config/boot.rb gem 'bootsnap', '>= 1.1.0', require: false # Use Pundit for access control. gem 'pundit', '~> 2.1' # Postgres Search gem 'pg_search', '~> 2.3' # Image transformation gem 'image_processing', '~> 1.12' # Validations for ActiveStorage. gem 'active_storage_validations', '~> 0.9.6' # Generate URL slugs for models, e.g. '/users/spiderman'. gem 'friendly_id', '~> 5.4.2' # Use SPARQL for querying Wikidata in imports. gem 'sparql', '~> 3.1.8', require: false # Use Addressable for use with the Wikidata API. gem 'addressable', '~> 2.8.0', require: false # Use the AWS SDK S3 gem for DigitalOcean Spaces - which are S3-compatible. gem 'aws-sdk-s3', '~> 1.66', require: false # A CLI progress bar for use with the import rake tasks. gem 'ruby-progressbar', '~> 1.11', require: false # Use Sentry for error tracking in production. gem 'sentry-ruby', '~> 4.8.1' gem 'sentry-rails', '~> 4.8.1' # Sorbet runtime typechecker and Sorbet Rails. gem 'sorbet-runtime', '~> 0.5' gem "sorbet-rails", '~> 0.7' # GraphQL API https://github.com/rmosolgo/graphql-ruby gem 'graphql', '~> 1.13.2' # Doorkeeper for OAuth API tokens gem "doorkeeper", "5.5.0.rc2" # Rack::Cors for handling CORS in API requests. gem "rack-cors", "~> 1.1" # Render SVGs inline using Webpacker. gem 'inline_svg', '~> 1.7' # Honeypot for spambots that doesn't require the user to actually do any # captcha and doesn't use google stuff. gem 'invisible_captcha', '~> 2.0.0' # For parallel execution of long-running tasks. gem 'parallel', '~> 1.21', require: false # For tracking changes to records. gem 'paper_trail', '~> 12.1' group :development, :test do # Call 'byebug' anywhere in the code to stop execution and get a debugger console gem 'byebug', platforms: [:mri, :mingw, :x64_mingw] # Pry for debugging. gem 'pry', '~> 0.14' gem 'pry-rails', '~> 0.3' # Rubocop for linting gem 'rubocop', '~> 1.24', require: false # rubocop extensions gem 'rubocop-performance', '~> 1.12', require: false gem 'rubocop-rspec', '~> 2.6', require: false gem 'rubocop-rails', '~> 2.12', require: false # Database cleaner for cleaning the database after tests/before seeding. gem 'database_cleaner', '~> 2.0' # Rspec-rails for testing. gem 'rspec-rails', '~> 5.0' # Factory Bot for creating factories. gem 'factory_bot_rails', '~> 6.2' # Code coverage gem 'simplecov', '~> 0.21', require: false # Shoulda-matchers for writing better tests on models. gem 'shoulda-matchers', '5.1.0' # For generating fake seeding data. gem 'faker', '~> 2.19' # For better display of rspec test suite progress gem 'fuubar', '~> 2.5.1' # Pundit matchers for simplifying policy testing. gem 'pundit-matchers', '~> 1.7.0' # Sorbet typechecker gem 'sorbet', '~> 0.5' end group :development do # Access an interactive console on exception pages or by calling 'console' anywhere in the code. gem 'web-console', '~> 4.2.0' gem 'listen', '~> 3.7' # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring gem 'spring', '~> 2.1' gem 'spring-watcher-listen', '~> 2.0.0' # Better error pages. gem 'better_errors', '~> 2.9' # For more useful BetterError error pages. gem 'binding_of_caller', '~> 1.0' # Open screenshots when they're taken with capybara. gem 'launchy', '~> 2.5' # Improve the formatting of GraphQL requests in the logs. # https://github.com/jetruby/graphql-rails_logger gem 'graphql-rails_logger', '~> 1.2.3' # Enable dotenv for local environment variables. gem 'dotenv-rails', '~> 2.7' # Tapioca for generating Sorbet RBI files. # Use a fork to fix this issue: https://github.com/Shopify/tapioca/issues/208 gem 'tapioca', '~> 0.6.0' end group :test do # Adds support for Capybara system testing and selenium driver gem 'capybara', '~> 3.36' gem 'selenium-webdriver', '~> 4.1' # Easy installation and use of WebDriver clients for various browsers. gem 'webdrivers', '~> 5.0' end # # Windows does not include zoneinfo files, so bundle the tzinfo-data gem # gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby] ```
Gemfile.lock ``` GEM remote: https://rubygems.org/ specs: actioncable (6.1.4.3) actionpack (= 6.1.4.3) activesupport (= 6.1.4.3) nio4r (~> 2.0) websocket-driver (>= 0.6.1) actionmailbox (6.1.4.3) actionpack (= 6.1.4.3) activejob (= 6.1.4.3) activerecord (= 6.1.4.3) activestorage (= 6.1.4.3) activesupport (= 6.1.4.3) mail (>= 2.7.1) actionmailer (6.1.4.3) actionpack (= 6.1.4.3) actionview (= 6.1.4.3) activejob (= 6.1.4.3) activesupport (= 6.1.4.3) mail (~> 2.5, >= 2.5.4) rails-dom-testing (~> 2.0) actionpack (6.1.4.3) actionview (= 6.1.4.3) activesupport (= 6.1.4.3) rack (~> 2.0, >= 2.0.9) rack-test (>= 0.6.3) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.0, >= 1.2.0) actiontext (6.1.4.3) actionpack (= 6.1.4.3) activerecord (= 6.1.4.3) activestorage (= 6.1.4.3) activesupport (= 6.1.4.3) nokogiri (>= 1.8.5) actionview (6.1.4.3) activesupport (= 6.1.4.3) builder (~> 3.1) erubi (~> 1.4) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.1, >= 1.2.0) active_storage_validations (0.9.6) activejob (>= 5.2.0) activemodel (>= 5.2.0) activestorage (>= 5.2.0) activesupport (>= 5.2.0) activejob (6.1.4.3) activesupport (= 6.1.4.3) globalid (>= 0.3.6) activemodel (6.1.4.3) activesupport (= 6.1.4.3) activerecord (6.1.4.3) activemodel (= 6.1.4.3) activesupport (= 6.1.4.3) activestorage (6.1.4.3) actionpack (= 6.1.4.3) activejob (= 6.1.4.3) activerecord (= 6.1.4.3) activesupport (= 6.1.4.3) marcel (~> 1.0.0) mini_mime (>= 1.1.0) activesupport (6.1.4.3) concurrent-ruby (~> 1.0, >= 1.0.2) i18n (>= 1.6, < 2) minitest (>= 5.1) tzinfo (~> 2.0) zeitwerk (~> 2.3) addressable (2.8.0) public_suffix (>= 2.0.2, < 5.0) amazing_print (1.4.0) ast (2.4.2) aws-eventstream (1.2.0) aws-partitions (1.390.0) aws-sdk-core (3.109.2) aws-eventstream (~> 1, >= 1.0.2) aws-partitions (~> 1, >= 1.239.0) aws-sigv4 (~> 1.1) jmespath (~> 1.0) aws-sdk-kms (1.39.0) aws-sdk-core (~> 3, >= 3.109.0) aws-sigv4 (~> 1.1) aws-sdk-s3 (1.83.2) aws-sdk-core (~> 3, >= 3.109.0) aws-sdk-kms (~> 1) aws-sigv4 (~> 1.1) aws-sigv4 (1.4.0) aws-eventstream (~> 1, >= 1.0.2) bcrypt (3.1.16) better_errors (2.9.1) coderay (>= 1.0.0) erubi (>= 1.0.0) rack (>= 0.9.0) bindex (0.8.1) binding_of_caller (1.0.0) debug_inspector (>= 0.0.1) bootsnap (1.9.3) msgpack (~> 1.0) builder (3.2.4) byebug (11.1.3) capybara (3.36.0) addressable matrix mini_mime (>= 0.1.3) nokogiri (~> 1.8) rack (>= 1.6.0) rack-test (>= 0.6.3) regexp_parser (>= 1.5, < 3.0) xpath (~> 3.2) childprocess (4.1.0) coderay (1.1.3) commander (4.6.0) highline (~> 2.0.0) concurrent-ruby (1.1.9) connection_pool (2.2.5) crass (1.0.6) database_cleaner (2.0.1) database_cleaner-active_record (~> 2.0.0) database_cleaner-active_record (2.0.1) activerecord (>= 5.a) database_cleaner-core (~> 2.0.0) database_cleaner-core (2.0.1) debug_inspector (1.1.0) devise (4.8.1) bcrypt (~> 3.0) orm_adapter (~> 0.1) railties (>= 4.1.0) responders warden (~> 1.2.3) diff-lcs (1.5.0) docile (1.4.0) doorkeeper (5.5.0.rc2) railties (>= 5) dotenv (2.7.6) dotenv-rails (2.7.6) dotenv (= 2.7.6) railties (>= 3.2) ebnf (2.2.1) amazing_print (~> 1.2) htmlentities (~> 4.3) rdf (~> 3.1) scanf (~> 1.0) sxp (~> 1.1) unicode-types (~> 1.6) erubi (1.10.0) factory_bot (6.2.0) activesupport (>= 5.0.0) factory_bot_rails (6.2.0) factory_bot (~> 6.2.0) railties (>= 5.0.0) faker (2.19.0) i18n (>= 1.6, < 2) faraday (1.8.0) faraday-em_http (~> 1.0) faraday-em_synchrony (~> 1.0) faraday-excon (~> 1.1) faraday-httpclient (~> 1.0.1) faraday-net_http (~> 1.0) faraday-net_http_persistent (~> 1.1) faraday-patron (~> 1.0) faraday-rack (~> 1.0) multipart-post (>= 1.2, < 3) ruby2_keywords (>= 0.0.4) faraday-em_http (1.0.0) faraday-em_synchrony (1.0.0) faraday-excon (1.1.0) faraday-httpclient (1.0.1) faraday-net_http (1.0.1) faraday-net_http_persistent (1.2.0) faraday-patron (1.0.0) faraday-rack (1.0.0) ffi (1.15.4) friendly_id (5.4.2) activerecord (>= 4.0.0) fuubar (2.5.1) rspec-core (~> 3.0) ruby-progressbar (~> 1.4) globalid (1.0.0) activesupport (>= 5.0) graphql (1.13.2) graphql-rails_logger (1.2.3) actionpack (> 5.0) activesupport (> 5.0) railties (> 5.0) rouge (~> 3.0) hamster (3.0.0) concurrent-ruby (~> 1.0) highline (2.0.3) htmlentities (4.3.4) i18n (1.8.11) concurrent-ruby (~> 1.0) image_processing (1.12.1) mini_magick (>= 4.9.5, < 5) ruby-vips (>= 2.0.17, < 3) inline_svg (1.7.2) activesupport (>= 3.0) nokogiri (>= 1.6) invisible_captcha (2.0.0) rails (>= 5.0) jbuilder (2.11.5) actionview (>= 5.0.0) activesupport (>= 5.0.0) jmespath (1.4.0) kaminari (1.2.1) activesupport (>= 4.1.0) kaminari-actionview (= 1.2.1) kaminari-activerecord (= 1.2.1) kaminari-core (= 1.2.1) kaminari-actionview (1.2.1) actionview kaminari-core (= 1.2.1) kaminari-activerecord (1.2.1) activerecord kaminari-core (= 1.2.1) kaminari-core (1.2.1) launchy (2.5.0) addressable (~> 2.7) link_header (0.0.8) listen (3.7.0) rb-fsevent (~> 0.10, >= 0.10.3) rb-inotify (~> 0.9, >= 0.9.10) logger (1.5.0) loofah (2.13.0) crass (~> 1.0.2) nokogiri (>= 1.5.9) mail (2.7.1) mini_mime (>= 0.1.1) marcel (1.0.2) matrix (0.4.2) method_source (1.0.0) mini_magick (4.11.0) mini_mime (1.1.2) mini_portile2 (2.6.1) minitest (5.15.0) msgpack (1.4.2) multipart-post (2.1.1) net-http-persistent (4.0.1) connection_pool (~> 2.2) nio4r (2.5.8) nokogiri (1.12.5) mini_portile2 (~> 2.6.1) racc (~> 1.4) nokogiri (1.12.5-x86_64-darwin) racc (~> 1.4) nokogiri (1.12.5-x86_64-linux) racc (~> 1.4) orm_adapter (0.5.0) paper_trail (12.1.0) activerecord (>= 5.2) request_store (~> 1.1) parallel (1.21.0) parlour (6.0.1) commander (~> 4.5) parser rainbow (~> 3.0) sorbet-runtime (>= 0.5) parser (3.0.3.2) ast (~> 2.4.1) pg (1.2.3) pg_search (2.3.5) activerecord (>= 5.2) activesupport (>= 5.2) polyfill (1.9.0) pry (0.14.1) coderay (~> 1.1) method_source (~> 1.0) pry-rails (0.3.9) pry (>= 0.10.4) public_suffix (4.0.6) puma (5.5.2) nio4r (~> 2.0) pundit (2.1.1) activesupport (>= 3.0.0) pundit-matchers (1.7.0) rspec-rails (>= 3.0.0) racc (1.6.0) rack (2.2.3) rack-cors (1.1.1) rack (>= 2.0.0) rack-proxy (0.7.0) rack rack-test (1.1.0) rack (>= 1.0, < 3) rails (6.1.4.3) actioncable (= 6.1.4.3) actionmailbox (= 6.1.4.3) actionmailer (= 6.1.4.3) actionpack (= 6.1.4.3) actiontext (= 6.1.4.3) actionview (= 6.1.4.3) activejob (= 6.1.4.3) activemodel (= 6.1.4.3) activerecord (= 6.1.4.3) activestorage (= 6.1.4.3) activesupport (= 6.1.4.3) bundler (>= 1.15.0) railties (= 6.1.4.3) sprockets-rails (>= 2.0.0) rails-dom-testing (2.0.3) activesupport (>= 4.2.0) nokogiri (>= 1.6) rails-html-sanitizer (1.4.2) loofah (~> 2.3) railties (6.1.4.3) actionpack (= 6.1.4.3) activesupport (= 6.1.4.3) method_source rake (>= 0.13) thor (~> 1.0) rainbow (3.0.0) rake (13.0.6) rb-fsevent (0.11.0) rb-inotify (0.10.1) ffi (~> 1.0) rbi (0.0.9) ast parser sorbet-runtime (>= 0.5.9204) unparser rdf (3.1.15) hamster (~> 3.0) link_header (~> 0.0, >= 0.0.8) rdf-aggregate-repo (3.1.0) rdf (~> 3.1) rdf-xsd (3.1.1) rdf (~> 3.1) rexml (~> 3.2) regexp_parser (2.2.0) request_store (1.5.0) rack (>= 1.4) responders (3.0.1) actionpack (>= 5.0) railties (>= 5.0) rexml (3.2.5) rouge (3.27.0) rspec-core (3.10.1) rspec-support (~> 3.10.0) rspec-expectations (3.10.1) diff-lcs (>= 1.2.0, < 2.0) rspec-support (~> 3.10.0) rspec-mocks (3.10.2) diff-lcs (>= 1.2.0, < 2.0) rspec-support (~> 3.10.0) rspec-rails (5.0.2) actionpack (>= 5.2) activesupport (>= 5.2) railties (>= 5.2) rspec-core (~> 3.10) rspec-expectations (~> 3.10) rspec-mocks (~> 3.10) rspec-support (~> 3.10) rspec-support (3.10.3) rubocop (1.24.0) parallel (~> 1.10) parser (>= 3.0.0.0) rainbow (>= 2.2.2, < 4.0) regexp_parser (>= 1.8, < 3.0) rexml rubocop-ast (>= 1.15.0, < 2.0) ruby-progressbar (~> 1.7) unicode-display_width (>= 1.4.0, < 3.0) rubocop-ast (1.15.0) parser (>= 3.0.1.1) rubocop-performance (1.12.0) rubocop (>= 1.7.0, < 2.0) rubocop-ast (>= 0.4.0) rubocop-rails (2.12.4) activesupport (>= 4.2.0) rack (>= 1.1) rubocop (>= 1.7.0, < 2.0) rubocop-rspec (2.6.0) rubocop (~> 1.19) ruby-progressbar (1.11.0) ruby-vips (2.1.4) ffi (~> 1.12) ruby2_keywords (0.0.5) rubyzip (2.3.2) safe_type (1.1.1) scanf (1.0.0) selenium-webdriver (4.1.0) childprocess (>= 0.5, < 5.0) rexml (~> 3.2, >= 3.2.5) rubyzip (>= 1.2.2) semantic_range (3.0.0) sentry-rails (4.8.1) railties (>= 5.0) sentry-ruby-core (~> 4.8.1) sentry-ruby (4.8.1) concurrent-ruby (~> 1.0, >= 1.0.2) faraday (>= 1.0) sentry-ruby-core (= 4.8.1) sentry-ruby-core (4.8.1) concurrent-ruby faraday shoulda-matchers (5.1.0) activesupport (>= 5.2.0) simplecov (0.21.2) docile (~> 1.1) simplecov-html (~> 0.11) simplecov_json_formatter (~> 0.1) simplecov-html (0.12.3) simplecov_json_formatter (0.1.3) sorbet (0.5.9464) sorbet-static (= 0.5.9464) sorbet-coerce (0.5.0) polyfill (~> 1.8) safe_type (~> 1.1, >= 1.1.1) sorbet-runtime (>= 0.4.4704) sorbet-rails (0.7.27) method_source (>= 0.9.2) parlour (>= 4.0.1) parser (>= 2.7) sorbet-coerce (>= 0.2.6) sorbet-runtime (>= 0.5) sorbet-runtime (0.5.9464) sorbet-static (0.5.9464-universal-darwin-14) sorbet-static (0.5.9464-universal-darwin-15) sorbet-static (0.5.9464-universal-darwin-16) sorbet-static (0.5.9464-universal-darwin-17) sorbet-static (0.5.9464-universal-darwin-18) sorbet-static (0.5.9464-universal-darwin-19) sorbet-static (0.5.9464-universal-darwin-20) sorbet-static (0.5.9464-universal-darwin-21) sorbet-static (0.5.9464-x86_64-linux) sparql (3.1.8) builder (~> 3.2) ebnf (~> 2.1) logger (~> 1.4) rdf (~> 3.1, >= 3.1.14) rdf-aggregate-repo (~> 3.1) rdf-xsd (~> 3.1) sparql-client (~> 3.1, >= 3.1.2) sxp (~> 1.1) sparql-client (3.1.2) net-http-persistent (~> 4.0, >= 4.0.1) rdf (~> 3.1) spoom (1.1.7) sorbet (>= 0.5.9204) sorbet-runtime (>= 0.5.9204) thor (>= 0.19.2) spring (2.1.1) spring-watcher-listen (2.0.1) listen (>= 2.7, < 4.0) spring (>= 1.2, < 3.0) sprockets (4.0.2) concurrent-ruby (~> 1.0) rack (> 1, < 3) sprockets-rails (3.4.2) actionpack (>= 5.2) activesupport (>= 5.2) sprockets (>= 3.0.0) sxp (1.1.0) rdf (~> 3.1) tapioca (0.6.0) bundler (>= 1.17.3) pry (>= 0.12.2) rbi (~> 0.0.0, >= 0.0.9) sorbet-runtime (>= 0.5.9204) sorbet-static (>= 0.5.9204) spoom (~> 1.1.0, >= 1.1.4) thor (>= 0.19.2) yard-sorbet thor (1.1.0) turbolinks (5.2.1) turbolinks-source (~> 5.2) turbolinks-source (5.2.0) tzinfo (2.0.4) concurrent-ruby (~> 1.0) unicode-display_width (2.1.0) unicode-types (1.7.0) unparser (0.6.2) diff-lcs (~> 1.3) parser (>= 3.0.0) warden (1.2.9) rack (>= 2.0.9) web-console (4.2.0) actionview (>= 6.0.0) activemodel (>= 6.0.0) bindex (>= 0.4.0) railties (>= 6.0.0) webdrivers (5.0.0) nokogiri (~> 1.6) rubyzip (>= 1.3.0) selenium-webdriver (~> 4.0) webpacker (5.4.3) activesupport (>= 5.2) rack-proxy (>= 0.6.1) railties (>= 5.2) semantic_range (>= 2.3.0) webrick (1.7.0) websocket-driver (0.7.5) websocket-extensions (>= 0.1.0) websocket-extensions (0.1.5) xpath (3.2.0) nokogiri (~> 1.8) yard (0.9.27) webrick (~> 1.7.0) yard-sorbet (0.6.1) sorbet-runtime (>= 0.5) yard (>= 0.9) zeitwerk (2.5.1) PLATFORMS ruby x86_64-darwin-18 x86_64-darwin-19 x86_64-darwin-20 x86_64-linux DEPENDENCIES active_storage_validations (~> 0.9.6) addressable (~> 2.8.0) aws-sdk-s3 (~> 1.66) better_errors (~> 2.9) binding_of_caller (~> 1.0) bootsnap (>= 1.1.0) byebug capybara (~> 3.36) database_cleaner (~> 2.0) devise (~> 4.8) doorkeeper (= 5.5.0.rc2) dotenv-rails (~> 2.7) factory_bot_rails (~> 6.2) faker (~> 2.19) friendly_id (~> 5.4.2) fuubar (~> 2.5.1) graphql (~> 1.13.2) graphql-rails_logger (~> 1.2.3) image_processing (~> 1.12) inline_svg (~> 1.7) invisible_captcha (~> 2.0.0) jbuilder (~> 2.11) kaminari (~> 1.2) launchy (~> 2.5) listen (~> 3.7) paper_trail (~> 12.1) parallel (~> 1.21) pg (~> 1.2) pg_search (~> 2.3) pry (~> 0.14) pry-rails (~> 0.3) puma (~> 5.5) pundit (~> 2.1) pundit-matchers (~> 1.7.0) rack-cors (~> 1.1) rails (~> 6.1.4) rspec-rails (~> 5.0) rubocop (~> 1.24) rubocop-performance (~> 1.12) rubocop-rails (~> 2.12) rubocop-rspec (~> 2.6) ruby-progressbar (~> 1.11) selenium-webdriver (~> 4.1) sentry-rails (~> 4.8.1) sentry-ruby (~> 4.8.1) shoulda-matchers (= 5.1.0) simplecov (~> 0.21) sorbet (~> 0.5) sorbet-rails (~> 0.7) sorbet-runtime (~> 0.5) sparql (~> 3.1.8) spring (~> 2.1) spring-watcher-listen (~> 2.0.0) tapioca (~> 0.6.0) turbolinks (~> 5) web-console (~> 4.2.0) webdrivers (~> 5.0) webpacker (~> 5.4.0) RUBY VERSION ruby 2.7.2p137 BUNDLED WITH 2.3.3 ```

Anyway, Merry Christmas and again - not urgent at all, just reporting it in case anyone else runs into it or these details help :)

EDIT: I removed these lines from my Gemfile.lock and it seems to be installing fine now:

    nokogiri (1.12.5-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.12.5-x86_64-linux)
      racc (~> 1.4)
psantos10 commented 2 years ago

Not urgent at all, obviously, since it's Christmas :)

Just wanted to note that I ran into this when trying to upgrade a project to Ruby 3.1 and running bundle install:

nokogiri-1.12.5-x86_64-darwin requires ruby version < 3.1.dev, >= 2.5, which is incompatible with the current version, ruby 3.1.0p0
% ruby -v
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin21]

The platforms in my Gemfile.lock include Ruby, so I'm not sure why Bundler isn't falling back to it automatically like (based on the description of this issue?) it should:

PLATFORMS
  ruby
  x86_64-darwin-18
  x86_64-darwin-19
  x86_64-darwin-20
  x86_64-linux

The following is my Gemfile and Gemfile.lock:

Gemfile Gemfile.lock Anyway, Merry Christmas and again - not urgent at all, just reporting it in case anyone else runs into it or these details help :)

EDIT: I removed these lines from my Gemfile.lock and it seems to be installing fine now:

    nokogiri (1.12.5-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.12.5-x86_64-linux)
      racc (~> 1.4)

@connorshea

1- Delete your Gemfile.lock 2 - run bundle install

Note that this will use ruby vanilha platform and not the native one.

deivid-rodriguez commented 2 years ago

We got a similar report on slack, bundler should be handling this fine, but this is tricky stuff and I believe we might not be there yet :( On an initial look, this would seem similar to https://github.com/rubygems/rubygems/issues/4455.

Does bundle update nokogiri work?

deivid-rodriguez commented 2 years ago

To clarify, Bundler should handle this just fine if there's no Gemfile.lock file. This problem is specific to having a lockfile generated with a previous Ruby, and thus including platform specific variants of nokogiri not compatible with the running Ruby. That's why @connorshea's smart trick works.

deivid-rodriguez commented 2 years ago

So, I tried to reproduce now and things seem to just work for me, but I tested this under arm64-darwin21, which is not locked at all, so I guess that's why.

@simi reported that somehow ruby version constraints were by passed for him and 1.12.5-x86_64-linux got installed under ruby 3.1. I guess that's what's causing the problem for him, but I still have no idea how 1.12.5-x86_64-linux would get installed under Ruby 3.1 in the first place.

connorshea commented 2 years ago

@deivid-rodriguez My project is open source, so you can try reproducing it yourself:

I'm on an M1 Mac, if that matters at all.

deivid-rodriguez commented 2 years ago

Yup,, I managed to reproduce easily with your initial Gemfile, but creating a lockfile from scratch with ruby 3.0 and then bundle it with ruby 3.1. I'm working on a fix now.

deivid-rodriguez commented 2 years ago

This is trickier than it seemed to me in the first place :(

When using gem information from the lockfile, it is initially forbidden to fetch the "real underlying specification" for each gem and we're only allowed to use names, versions, and dependency information recorded in the lockfile. The problem is, required_ruby_version metadata is not recorded in the lockfile, so we can't check it before resolving in order to figure out whether we need to discard some data in the lockfile due to not satisfying the running ruby version.

To fix the issue I can think of two approaches:

I'll try hard to figure out some solution.

connorshea commented 2 years ago

Thank you for all your great work as always, Deivid 🙇‍♂️

mockdeep commented 2 years ago

Oddly, I ran into this issue with only ruby in the PLATFORMS section. When I deleted the Gemfile.lock and ran bundle it updated PLATFORMS to be just x86_64-linux.

deivid-rodriguez commented 2 years ago

Yes, I think that is expected. Even if your lockfile is locked to RUBY, bundler will still try to also resolve for your specific platform, and will run into the same issue.

skaes commented 2 years ago

In my experience, the only reliable way to avoid the aforementioned problems is to use bundle config set force_ruby_platform true.

sergiopantoja commented 2 years ago

I had a similar experience as @mockdeep. I deleted my Gemfile.lock, ran bundle, and it completed successfully on a Macbook Air M1. My Gemfile.lock diff afterwards was:

 PLATFORMS
-  ruby
+  arm64-darwin-21
deivid-rodriguez commented 2 years ago

Has someone tried bundle update nokogiri? I just tried it myself and it worked fine, since it completely avoids the lockfile for nokogiri.

bak commented 2 years ago

On an intel mac, with several rails applications, deleting the Gemfile.lock (or bypassing with bundle update nokogiri) didn't work for me. Following @skaes' advice, however, has produced clean builds with no lockfile diff.

deivid-rodriguez commented 2 years ago

Really? What error did you get? That's completely unexpected and sounds like a similar issue to the one reported by @simi in Slack. If you were able to reproduce that and report an issue to https://github.com/rubygems/rubygems, that'd be great.

bak commented 2 years ago

@deivid-rodriguez the error I was getting from running bundle install against an existing Gemfile was:

nokogiri-1.12.5-x86_64-darwin requires ruby version >= 2.5, < 3.1.dev, which is incompatible with the current version, ruby 3.1.0p0

Which is oddly very close to but not exactly what @connorshea reported above (the version bounds are reversed—they had < 3.1.dev, >= 2.5).

My Gemfile's platforms were simply:

PLATFORMS
  ruby
simi commented 2 years ago

This is exactly my problem, I'll try to dig deeper soon to provide reproducing script.

bak commented 2 years ago

I'm able to repro with a Gemfile that simply contains:

source "https://rubygems.org"
gem "nokogiri"

Here's a transcript of working with it:

14:56:41 ~/work/etc/bundletest/ % ls
Gemfile
14:56:50 ~/work/etc/bundletest/ % cat Gemfile
source "https://rubygems.org"
gem "nokogiri"
14:56:59 ~/work/etc/bundletest/ % ruby -v
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
14:57:03 ~/work/etc/bundletest/ % gem -v
3.3.3
14:57:13 ~/work/etc/bundletest/ % bundle config
Settings are listed in order of priority. The top value will be used.
14:57:21 ~/work/etc/bundletest/ % bundle
Fetching gem metadata from https://rubygems.org/.......
Resolving dependencies...
nokogiri-1.12.5-x86_64-darwin requires ruby version >= 2.5, < 3.1.dev, which is incompatible with the current version, ruby 3.1.0p0
14:57:28 ~/work/etc/bundletest/ % bundle config set force_ruby_platform true
14:57:33 ~/work/etc/bundletest/ % bundle
Fetching gem metadata from https://rubygems.org/.......
Resolving dependencies...
Using bundler 2.3.3
Using racc 1.6.0
Using mini_portile2 2.6.1
Using nokogiri 1.12.5
Bundle complete! 1 Gemfile dependency, 4 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
14:57:40 ~/work/etc/bundletest/ % cat Gemfile.lock
GEM
  remote: https://rubygems.org/
  specs:
    mini_portile2 (2.6.1)
    nokogiri (1.12.5)
      mini_portile2 (~> 2.6.1)
      racc (~> 1.4)
    racc (1.6.0)

PLATFORMS
  ruby

DEPENDENCIES
  nokogiri

BUNDLED WITH
   2.3.3
15:00:15 ~/work/etc/bundletest/ % bundle config set force_ruby_platform false
You are replacing the current global value of force_ruby_platform, which is currently "true"
15:00:26 ~/work/etc/bundletest/ % bundle
Using bundler 2.3.3
Using mini_portile2 2.6.1
Using racc 1.6.0
Using nokogiri 1.12.5 (x86_64-darwin)
Bundle complete! 1 Gemfile dependency, 4 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

One thing that jumps out at me is, after I set force_ruby_platform to false, the install succeeds again but as Using nokogiri 1.12.5 **(x86_64-darwin)**

@deivid-rodriguez apologies, I'm not sure exactly how to frame this for a rubygems issue. What other info could I provide?

deivid-rodriguez commented 2 years ago

So @simi's issue was that somehow nokogiri-1.12.5-x86_64-linux got installed in his system on Ruby 3.1, which is completely unexpected since it's not supported there. Somehow required_ruby_version constraints were bypassed but we don't know how yet. I think your issue is the same, but on MacOS. If you manually uninstall nokogiri-1.12.5-x86_64-darwin through gem uninstall nokogiri, it should work again.

The problem, and what we need to be able to reproduce reliably, is how this is happening in the first place.

...

Actually, while I was writing this I think I realized what the problem might be. Let me do some testing and I'll get back to you.

deivid-rodriguez commented 2 years ago

No, not really, I'm unable to reproduce this :(

bak commented 2 years ago

Yep you're right, after doing gem uninstall nokogiri (and "yes"ing everything), it behaves great.

bak commented 2 years ago

@deivid-rodriguez I think I've got something. My initial bundle install with Ruby 3.1.0 was on a project that used a private gem repository (Artifactory) instead of rubygems. Redoing the basic Gemfile above allows the platform-specific version to be installed:

16:06:56 ~/work/etc/bundletest/ % gem list | grep nokogiri
16:07:00 ~/work/etc/bundletest/ % ruby -v
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
16:07:04 ~/work/etc/bundletest/ % gem -v
3.3.3
16:07:07 ~/work/etc/bundletest/ % bundler -v
Bundler version 2.3.3
16:07:12 ~/work/etc/bundletest/ % l
total 16
-rw-r--r--  1 bencullen-kerney  11 Dec 29 14:37 .ruby-version
-rw-r--r--  1 bencullen-kerney  84 Dec 29 16:06 Gemfile
16:07:18 ~/work/etc/bundletest/ % cat Gemfile
source "https://XXXXXXX/"
gem "nokogiri"
16:07:23 ~/work/etc/bundletest/ % bundle
Fetching gem metadata from https://XXXXXXX/........
Resolving dependencies...
Using bundler 2.3.3
Using racc 1.6.0
Fetching nokogiri 1.12.5 (x86_64-darwin)
Installing nokogiri 1.12.5 (x86_64-darwin)
Bundle complete! 1 Gemfile dependency, 3 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
16:07:45 ~/work/etc/bundletest/ % gem list | grep nokogiri
nokogiri (1.12.5 x86_64-darwin)
16:07:58 ~/work/etc/bundletest/ % cat Gemfile.lock
GEM
  remote: https://XXXXXXX/
  specs:
    nokogiri (1.12.5-x86_64-darwin)
      racc (~> 1.4)
    racc (1.6.0)

PLATFORMS
  x86_64-darwin-20

DEPENDENCIES
  nokogiri

BUNDLED WITH
   2.3.3

Unfortunately I'm not very familiar with how the repository is configured, but happy to dig in where I can.

bak commented 2 years ago

Also apologies for continuing this thread—this is clearly not a nokogiri-specific bug. I'm honestly not sure it's a rubygems bug, or if the private repository is misconfigured somehow.

simi commented 2 years ago

My problem was with Geminabox private repo, but I have seen it using rubygem.org as well.

bak commented 2 years ago

@simi did you gem uninstall nokogiri between the two? It seems that, once the platform-specific version is allowed through, it will reproduce for Gemfiles with the rubygems source.

deivid-rodriguez commented 2 years ago

Yeah, maybe this should be moved to our repository. I think the culprit is here:

https://github.com/rubygems/rubygems/blob/62d14e47f58d8cc3c32f0e69a082c7d9a16a001e/bundler/lib/bundler/lazy_specification.rb#L92-L96

The actual filtering of required_ruby_version is only done for EndpointSpecifications, which represent the data given by the Compact Index API. Older APIs provide RemoteSpecification objects, which did not include required_ruby_version information. And custom gem servers like Geminabox only support older APIs as far as I know.

mockdeep commented 2 years ago

Has someone tried bundle update nokogiri?

I tried that and it didn't fix the issue. I needed to regenerate Gemfile.lock.

flavorjones commented 2 years ago

Hi, although this conversation is valuable, I'd prefer if it didn't happen here and instead happened on a bundler issue. @deivid-rodriguez can you recommend somewhere this can continue?

Please don't run bundle config set force_ruby_platform true to work around problems. Instead uninstall all nokogiri versions from your system and re-bundle.

flavorjones commented 2 years ago

See #2408 for status on shipping 3.1 support in native precompiled gem releases.

deivid-rodriguez commented 2 years ago

@flavorjones Yes, I'm so sorry for hijacking this thread with a bundler issue. I created a fresh ticket in our repository: https://github.com/rubygems/rubygems/issues/5251. Let's post any comments about that issue there.

MtnBiker commented 2 years ago

I hope this isn't adding noise. On MBP M1. No Gemfile.lock file has yet been created. New Rail 7 app

➜ gem install nokogiri
Building native extensions. This could take a while...
Successfully installed nokogiri-1.12.5
1 gem installed

app_name on  main [?] via 💎 ruby-3.1.0 took 42s
➜ bundle update
Fetching gem metadata from https://rubygems.org/...........
Resolving dependencies...
nokogiri-1.12.5-arm64-darwin requires ruby version >= 2.5, < 3.1.dev, which is incompatible with the current version, ruby 3.1.0p0

Tried

➜ bundle update nokogiri
This Bundle hasn't been installed yet. Run `bundle install` to update and install the bundled gems.

Thanks everyone for working on this

flavorjones commented 2 years ago

@MtnBiker please uninstall all versions of nokogiri and try again.

flavorjones commented 2 years ago

@MtnBiker also please make sure "ruby" is a platform that bundler is including in your Gemfile.lock under the PLATFORMS section. Use bundler lock --add-platform ruby if you need to.

MtnBiker commented 2 years ago

@flavorjones and others. Thank you. I sorted it out. I think the problem was that Rails 7.0.0 wasn't compatible with Ruby 3.1.0. So went with another version of Ruby. Ran into another problem with pg. Lots of activity on Rails, but in response tot the 3.1.0 problem: "This is already fixed on main - #43951. It was expected for rails 7.0.1 to be released before ruby 3.1.0. See https://bugs.ruby-lang.org/issues/14394#note-36." But current release still at 7.0.0. And mentions of Ruby 3.2 in Issues

jrochkind commented 1 year ago

Oh that nokogiri issue is about ruby 3.1, from a year ago, not ruby 3.2. Still, it explains how nokogiri is "supposed" to work, and combined with the 1.14.0.rc1 release notes -- I think it is exactly what is going on now with ruby 3.2, it explains everything else I've been seeing.

Everywhere else, bundler figures out not to use the pre-compiled nokogiri, which is why I haven't had any problems with it in those other environments -- and noticed nokogiri taking longer to install, compiling from source!

But on samvera CircleCI builds, nope. And I suspect the custom caching routines in our samvera orb.