bundle path issue with racc - preventing to use openstudio extension gem's openstudio:test_with_openstudio.

jmarrec commented 5 months ago

Issue overview

We have gems that indirectly depend on racc. I can't seem to run with a bundle / Gemfile that has racc as a dependency on either of my machines, but it works on nrel/openstudio:3.7.0 docker, why?

rubocop 1.15.0 (and nokogiri and rubyXL) depend on racc.

example

https://github.com/NREL/OpenStudio-measure-tester-gem/blob/14659ef904019789f651a890bc56d2324d725805/openstudio_measure_tester.gemspec#L37

Current Behavior

echo "source 'https://rubygems.org'" > Gemfile && echo "gem 'racc', '1.7.2'" >> Gemfile
bundle install --path=.bundle
openstudio --verbose --bundle Gemfile --bundle_path ./.bundle --bundle_without native_ext gem_list

$ openstudio classic --verbose --bundle Gemfile --bundle_path ./.bundle --bundle_without native_ext gem_list
D, [2024-02-15T21:51:25.543493 #58828] DEBUG -- : Adding dependency on racc '~> 1.7.2'

D, [2024-02-15T21:51:25.564884 #58828] DEBUG -- : Processing 61 activation requests
D, [2024-02-15T21:51:25.566216 #58828] DEBUG -- : Activating gem :/ruby/2.7.0/specifications/bundler-2.1.4.gemspec
I, [2024-02-15T21:51:25.968700 #58828]  INFO -- : without_groups = native_ext
I, [2024-02-15T21:51:25.968732 #58828]  INFO -- : g = default
I, [2024-02-15T21:51:25.968741 #58828]  INFO -- : Bundling with groups [default]
Error executing argv: ["--verbose", "--bundle", "Gemfile", "--bundle_path", "./.bundle", "--bundle_without", "native_ext", "gem_list"]
Error: Could not find racc-1.7.2 in any of the sources in :/ruby/2.7.0/gems/bundler-2.1.4/lib/bundler/spec_set.rb:86:in `block in materialize'
:/ruby/2.7.0/gems/bundler-2.1.4/lib/bundler/spec_set.rb:80:in `map!'
:/ruby/2.7.0/gems/bundler-2.1.4/lib/bundler/spec_set.rb:80:in `materialize'
:/ruby/2.7.0/gems/bundler-2.1.4/lib/bundler/definition.rb:170:in `specs'
:/ruby/2.7.0/gems/bundler-2.1.4/lib/bundler/definition.rb:237:in `specs_for'
:/openstudio_cli.rb:654:in `parse_main_args'
:/openstudio_cli.rb:784:in `execute'
:/openstudio_cli.rb:1987:in `<main>'
eval:193:in `eval'
eval:193:in `require_embedded_absolute'
eval:178:in `block in require_embedded'
eval:172:in `each'
eval:172:in `require_embedded'
eval:131:in `require'
eval:3:in `<main>'

Expected Behavior

Steps to Reproduce

On my M1 mac and on my Ubuntu 20.04 it fails regardless of whether I try racc 1.7.2 (like the embedded one) or 1.7.3. I use an RVM provided ruby on both, but I've tried manually unsetting GEM_HOME / GEM_PATH (even though the embedded ruby does that already)

On docker, docker run -it --rm --platform linux/amd64 nrel/openstudio:3.7.0 it works with 1.7.2 and with 1.7..3

root@255a690713d7:/var/simdata/openstudio# openstudio --verbose --bundle Gemfile --bundle_path ./.bundle --bundle_without native_ext gem_list
Setting Log Level to Debug (-2)
[ApplicationPathHelpers] <-2> getOpenStudioModule, info.dli_fname = '/usr/local/openstudio-3.7.0/bin/../lib/libopenstudiolib.so'
[PathHelpers] <-2> completeAndNormalize: result = /usr/local/openstudio-3.7.0/lib/libopenstudiolib.so
[ApplicationPathHelpers] <-2> getOpenStudioModule, found '/usr/local/openstudio-3.7.0/lib/libopenstudiolib.so'
bundler (2.1.4) ':/ruby/2.7.0/gems/bundler-2.1.4'
racc (1.7.2) '/var/simdata/openstudio/.bundle/ruby/2.7.0/gems/racc-1.7.2'

Possible Solution

Details

Environment

Some additional details about your environment for this issue (if relevant):

Platform (Operating system, version):
Version of OpenStudio (if using an intermediate build, include SHA):

Context

jmarrec commented 5 months ago

openstudio -e "puts EmbeddedScripting::getFileAsString(':/ruby/2.7.0/specifications/racc-1.7.2.gemspec')"

# -*- encoding: utf-8 -*-
# stub: racc 1.7.2 ruby lib
# stub: ext/racc/cparse/extconf.rb

Gem::Specification.new do |s|
  s.name = "racc".freeze
  s.version = "1.7.2"

  s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
  s.require_paths = ["lib".freeze]
  s.authors = ["Minero Aoki".freeze, "Aaron Patterson".freeze]
  s.date = "2023-11-01"
  s.description = "Racc is a LALR(1) parser generator.\n  It is written in Ruby itself, and generates Ruby program.\n\n  NOTE: Ruby 1.8.x comes with Racc runtime module.  You\n  can run your parsers generated by racc 1.4.x out of the\n  box.\n".freeze
  s.email = [nil, "aaron@tenderlovemaking.com".freeze]
  s.executables = ["racc".freeze]
  s.extensions = ["ext/racc/cparse/extconf.rb".freeze]
  s.extra_rdoc_files = ["README.ja.rdoc".freeze, "README.rdoc".freeze]
  s.files = ["README.ja.rdoc".freeze, "README.rdoc".freeze, "bin/racc".freeze, "ext/racc/cparse/extconf.rb".freeze]
  s.homepage = "https://github.com/ruby/racc".freeze
  s.licenses = ["Ruby".freeze, "BSD-2-Clause".freeze]
  s.rdoc_options = ["--main".freeze, "README.rdoc".freeze]
  s.required_ruby_version = Gem::Requirement.new(">= 2.5".freeze)
  s.rubygems_version = "3.1.4".freeze
  s.summary = "Racc is a LALR(1) parser generator".freeze

  s.installed_by_version = "3.1.4" if s.respond_to? :installed_by_version
end

openstudio -e "puts Gem::Specification.map{|spec| [spec.name, spec.version].join('-')}"
openstudio -e "p Gem::Specification.select{|spec| spec.name == 'racc'}"

jmarrec commented 5 months ago

WOrks on docker nrel/openstudio:3.7.0 (which is an ubuntu 20.04) and fails on my machine... env -i clears the entire environment variables.

env -i /usr/local/openstudio-3.7.0/bin/openstudio classic --verbose --bundle Gemfile --bundle_path ./.bundle --bundle_without native_ext -e "puts ENV.to_h"

jmarrec commented 5 months ago

Seems like pinning parser to 3.2.2.2 is enough to Not have racc as a runtime dependency:

https://github.com/whitequark/parser/commit/37146d8247b02dfcd9c94163384b1069d8e0a3a6

THis is done on develop for OpenStudio-measure-tester-gem but I don't think there are any released version with it.

https://github.com/NREL/OpenStudio-measure-tester-gem/commit/60ba1ad63c23194ff3eb2124574efc96d0a33ccf

and https://github.com/NREL/OpenStudio-measure-tester-gem/issues/66

jmarrec commented 5 months ago

Still going to have issues since some of the OS gems depend on stuff like rubyXL or Nokogiri which in turn depend on racc.

We'll need to figure out what's the problem with racc.

jmarrec commented 3 months ago

With ruby3 gems update, racc is not longer a dependency we have (measue-tester-gem pinned parser to a version before it was introduced)

NREL / OpenStudio