Error when using exec binding to call external ruby scripts

[pacive]

I just encountered a strange bug when using the exec binding to call a ruby script, which seems to have something to to with bundler. As a simple test case this rule can be used:

require 'openhab'

rule 'Test' do
  on_start
  run do
    logger.info("Helper library version: #{OpenHAB::VERSION}")
    logger.info(Exec.executeCommandLine(5.seconds, 'ruby', '-e', "puts 'test'"))
  end 
end

Which logs:

2022-08-26 14:50:45.998 [INFO ] [g.openhab.automation.jruby.test.test] - Helper library version: 4.44.1
2022-08-26 14:50:46.229 [INFO ] [g.openhab.automation.jruby.test.test] - <internal:/usr/local/rvm/rubies/ruby-3.0.0/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/bundler/setup (LoadError)
    from <internal:/usr/local/rvm/rubies/ruby-3.0.0/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

Running the command directly at the terminal (using the openhab user) works as expected, the issue only appears when doing it in a rule.

As indicated above I have ruby 3.0 installed by using rvm.

[ccutrer]

Yes, it's an issue with nested bundler. To address it, run your script like this:

    Bundler.with_clean_env do
      holiday = `/var/lib/openhab/holidays.rb`.strip
    end

[ccutrer]

Oh, I should mention this should work regardless of how the sun-Ruby is spawned - backticks like I have here (to capture the output), the system call, some form of popen, or the exec action like you're using. I stopped using the exec action when I realized it was much easier to just use Ruby methods to spawn new processes, and my old DSL rules only used exec because there was no "native" way to do it.

[pacive]

Yes, it's an issue with nested bundler. To address it, run your script like this:

    Bundler.with_clean_env do
      holiday = `/var/lib/openhab/holidays.rb`.strip
    end

Did something change regarding bundler recently? Because it has always worked before. I just updated to 3.4.M1 today, which probably updated the helper library as well when restarting, so not sure if it's in the library or the jruby addon.

[ccutrer]

You probably got a jrubyscripting commit that uses bundler to install openhab-scripting gem, but that shouldn't change this behavior (since that happens in a completely separate instance of the JRuby engine). This is because OpenHAB-scripting requires bundler/inline, regardless of if you actually use it. It's done that for as long as I've been involved in the project.

[pacive]

Just tested, and confirmed that this doesn't happen on OH 3.3 stable. I noticed that bundler is not installed as a gem on that version, so I suspect that this change caused a regression.

Edit: should I post this as an issue on openhab-addons instead?

[pacive]

Took a look at the environment for the shell that gets created and noticed this environment variable:

RUBYOPT=-ruri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/bundler/setup

Unsetting it by running the command as:

logger.info(Exec.executeCommandLine(5.seconds, 'env', '-u', 'RUBYOPT', 'ruby', '-e', "puts 'test'"))

made it work as expected. Can this be unset by the addon or the helper library to prevent this?

[boc-tothefuture]

Just tested, and confirmed that this doesn't happen on OH 3.3 stable. I noticed that bundler is not installed as a gem on that version, so I suspect that this change caused a regression.

That line isn't installing bundler, it is just requiring it. The scripting gem does have bundler as a dependency so when you install the scripting gem it installs bundler. That was add here in Jan of 2021.

[pacive]

Just tested, and confirmed that this doesn't happen on OH 3.3 stable. I noticed that bundler is not installed as a gem on that version, so I suspect that this change caused a regression.

That line isn't installing bundler, it is just requiring it.

Ah, sorry, read it too fast. But then I would definitely say that this is probably what's changing the environment.

[boc-tothefuture]

I don't know why that would change the environment.. It is run with engine.eval during binding startup and then disposed of. It could also be something changed in JRuby itself that that environment var.

[pacive]

Why would JRuby set an environment variable to always require bundler/setup?

[boc-tothefuture]

Just in how rubyopt is persisted across sessions.. I am really guessing at this point.. :-)

[boc-tothefuture]

In this gem we have always (as far as I know) done that same require bundler/inline in our code. So if it causes a global change that should have always been an issue.

[boc-tothefuture]

Do you hit the same issue on 3.3 with the latest library? That might help narrow it down for us.

[boc-tothefuture]

Yes, it's an issue with nested bundler. To address it, run your script like this:

    Bundler.with_clean_env do
      holiday = `/var/lib/openhab/holidays.rb`.strip
    end

Would it make sense for us to monkeypatch exec to have that built in?

[ccutrer]

It's definitely bundler setting it:

https://github.com/rubygems/rubygems/blob/773fc2b9287ef06b5674dcac9b704aaba73481d6/bundler/lib/bundler/shared_helpers.rb#L297

It's triggered when you actually call gemfile, not just require "bundler/inline", which explains why I've always seen it, but @pacive has not.

I'm just not sure how it's leaking between JRuby script containers.

[ccutrer]

And here it is - jruby defaults to updating its parent process's ENV: https://github.com/jruby/jruby/blob/a0dea0a4084032e5941977d5a1f105c2acb0ab29/core/src/main/java/org/jruby/RubyInstanceConfig.java#L1557

[ccutrer]

I just pushed a fix to openhab-addons.

[pacive]

Great work, thank you!

[pacive]

Fixed by openhab/openhab-addons#13328