sparklemotion / nokogiri

Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.
https://nokogiri.org/
MIT License
6.15k stars 897 forks source link

Updating the psych requirements to 5.0 breaks macOS build #2769

Closed stevecheckoway closed 1 year ago

stevecheckoway commented 1 year ago

Commit c7a780298614c81f3dc5925e2b0c23b836e1c6e0 updates the psych requirements from 4.0 to 5.0 and this is broken on macOS. There's more context here.

The basic problem is that psych no longer bundles libyaml and macOS doesn't come with libyaml nor does it have a system package manager that can be used to install it. It seems the expected situation is to use homebrew to install the library system wide.

It's possible to install the psych gem by passing an option pointing to the libyaml source directory, but this is broken on M1 macs. After I filed a bug report, a pre-release version of psych was created and pushed to rubygems. I can successfully install this version of psych, but that doesn't help with building Nokogiri any longer as bundle install still tries to install the non-prerelease version of the gem.

bundle install output ``` $ bundle install Fetching gem metadata from https://rubygems.org/........ Resolving dependencies... Using rake 13.0.6 Using ast 2.4.2 Using builder 3.2.4 Using bundler 2.3.7 Using docile 1.4.0 Using hoe-markdown 1.4.0 Using mini_portile2 2.8.1 Using minitest 5.17.0 Using ruby-progressbar 1.11.0 Using racc 1.6.2 Using rexml 3.2.5 Using parallel 1.22.1 Using rake-compiler 1.2.1 Using rake-compiler-dock 1.3.0 Using regexp_parser 2.6.1 Using nokogiri 1.14.1.dev from source at `.` and installing its executables Using parser 3.2.0.0 Using rainbow 3.1.1 Using rubocop-ast 1.24.1 Using json 2.6.3 Using stringio 3.0.4 Using simplecov-html 0.12.3 Using simplecov_json_formatter 0.1.4 Fetching psych 5.0.1 Using ansi 1.5.0 Using rexical 1.0.7 Using unicode-display_width 2.4.2 Using minitest-reporters 1.5.0 Using simplecov 0.21.2 Using rubocop 1.43.0 Using rubocop-rake 0.6.0 Using rubocop-performance 1.15.2 Using rubocop-shopify 2.10.1 Using rubocop-minitest 0.26.0 Using ruby_memcheck 1.2.0 Installing psych 5.0.1 with native extensions Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /Users/steve/.local/share/gem/ruby/3.1.0/gems/psych-5.0.1/ext/psych /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/bin/ruby -I /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0 extconf.rb checking for yaml.h... no yaml.h not found *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/bin/$(RUBY_BASE_NAME) --with-libyaml-source-dir --without-libyaml-source-dir --with-yaml-0.1-dir --without-yaml-0.1-dir --with-yaml-0.1-include --without-yaml-0.1-include=${yaml-0.1-dir}/include --with-yaml-0.1-lib --without-yaml-0.1-lib=${yaml-0.1-dir}/lib --with-yaml-0.1-config --without-yaml-0.1-config --with-pkg-config --without-pkg-config --with-libyaml-dir --without-libyaml-dir --with-libyaml-include --without-libyaml-include=${libyaml-dir}/include --with-libyaml-lib --without-libyaml-lib=${libyaml-dir}/lib To see why this extension failed to compile, please check the mkmf.log which can be found here: /Users/steve/.local/share/gem/ruby/3.1.0/extensions/arm64-darwin-22/3.1.0/psych-5.0.1/mkmf.log extconf failed, exit code 1 Gem files will remain installed in /Users/steve/.local/share/gem/ruby/3.1.0/gems/psych-5.0.1 for inspection. Results logged to /Users/steve/.local/share/gem/ruby/3.1.0/extensions/arm64-darwin-22/3.1.0/psych-5.0.1/gem_make.out /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:102:in `run' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/ext_conf_builder.rb:28:in `build' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:171:in `build_extension' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:205:in `block in build_extensions' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:202:in `each' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/ext/builder.rb:202:in `build_extensions' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/rubygems/installer.rb:843:in `build_extensions' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/rubygems_gem_installer.rb:71:in `build_extensions' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/rubygems_gem_installer.rb:28:in `install' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/source/rubygems.rb:204:in `install' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/installer/gem_installer.rb:54:in `install' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/installer/gem_installer.rb:16:in `install_from_spec' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/installer/parallel_installer.rb:186:in `do_install' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/installer/parallel_installer.rb:177:in `block in worker_pool' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/worker.rb:62:in `apply_func' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/worker.rb:57:in `block in process_queue' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/worker.rb:54:in `loop' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/worker.rb:54:in `process_queue' /nix/store/4a5x886ahr2g90yknfw9y7pnn63zn85l-ruby-3.1.2/lib/ruby/3.1.0/bundler/worker.rb:91:in `block (2 levels) in create_threads' An error occurred while installing psych (5.0.1), and Bundler cannot continue. In Gemfile: rdoc was resolved to 6.5.0, which depends on psych ```

This absolutely seems like an upstream issue but given that they decided to stop bundling libyaml, it seems unlikely they're going to fix it.

I tried using rvm but—after undoing all the damage the installer does to my shell dot files—it turns out that rvm depends on a global installation of homebrew.

Suggestions welcome.

flavorjones commented 1 year ago

@stevecheckoway Yeah, sorry for the trouble. Psych 5 is indeed a challenging thing to install. We have a few options ...

The easiest thing to do might be to edit your local Gemfile for now to remove Psych. You may still need to explicitly pin it to ~> 4.0 to prevent an upgrade to Psych 5, though.

So the next option might be to remove both rdoc and psych from your Gemfile, then you'll get and use whatever version your Ruby was shipped with. If that works for you, I can create a bundler "group" for documentation and you can turn it off permanently on your local machine. Want to give that a try with me?

stevecheckoway commented 1 year ago

@flavorjones Making a new bundler group was a great idea. I posted a PR that does that.

flavorjones commented 1 year ago

Closed via #2770