Rjb hangs in Ruby 2.0 within background jobs

[ryanjones]

I've tried making a call through Rjb in both Sidekiq and Resque. When the background job is being processed it gets up to .call and then just hangs.

This only happens in Ruby 2.0 (Ruby 1.9.3 seems to work fine). I've tried with both Rjb 1.4.3 & 1.4.8.

The Rjb code works fine within an inline process (a page request in rails).

Other people seem to be running into a similar problem here: https://github.com/resque/resque/issues/795

[arton]

I have no knowledge about both Sidekiq and Resque, so I wonder if you let me know the technical details of 'background job'. Is it a forked and executed process ? If so, the hangup means 1) the parent process can't recognize the termination of the child process or 2) the child process runs eternally ? And it's very important that what function Sidekiq or Resque use to spawn the job. Kenrl.fork, system or spawn ? Because Ruby was restructed these functions between 1.9 and 2.0.

[ryanjones]

Resque forks: https://github.com/resque/resque/blob/6afe925e8bb6376f20d143c9b40c05b7f53bbf89/lib/resque/child_process.rb#L54

Sidekiq uses celluloid (threads): https://github.com/celluloid/celluloid/blob/ebd0c300ea76b43c1881c1036f918133b6bf7b6a/lib/celluloid/internal_pool.rb#L97

It's almost like the child process runs until rescue/sidekiq kill the job. Through logging in 1.9.3 we had some logging before and after the call (which worked correctly). In 2.0 it did the logging before the call, and then the call just hung.

I can't seem to dig up a good resource on the changes between 1.9 and 2.0 Kernel/Thread, do you have one that I could take a look at? I'd like to contribute backwards compatibility between 1.9 and 2.0.

We had to write a quick rake task to deal with our Rjb job, and I'd really like to move that into a proper background job processing system.

[arton]

Umm, it's difficult because I see that Resque and Sidekiq use completely separate issues. Would you get Rjb's verbose message with -d and -w options on command line or set $DEBUG = true ; $VERBOSE = true ? If the options or global variables set to true, Rjb emits its errors on to $STDOUT. And please let me know what Java classes you use in the script. Can you reproduce the issue with only very simple Java object ssuch as java.lang.String etc ?

[arton]

No logs ?

[poori]

I have a bit more information about this (ran into the same thing)

My environment: rails 4, ruby 2.1.1p76,

class SimpleJob
  include Resque::Plugins::Status
  @queue = :fetch_scheduler

  $DEBUG=true #is this how you set it? I don't know...
  $VERBOSE = true
  require 'rjb'
  Point = Rjb::import('java.awt.Point')

  def perform
    p = Point.new(0, 0)
    p.y = 80
    puts "x=#{p.x}, y=#{p.y}" # => "0,80"
  end

  end

the above works fine. However, calling my intended java library, it goes into a black hole and does not return. If you can be more specific in how to turn on the verbose msg options, I can try those.

I can confirm it also runs fine in an inline process.

[arton]

It seems PATH or version conflicting issue. I wonder if you redirected the stdout to a file for example invoked it by shell script. I strongly recommand that if you use rvm or rbenv, re install gems by each environment and each separated directries. Regards

[halida]

Same issue, Under resque worker, Rjb::import('java.awt.Point') works fine, but when load other jar and import other class cause trouble, like this:

Rjb::load('/Users/halida/data/workspace/sources/pdf-stamper/ext/iText-4.2.0.jar', ['-Djava.awt.headless=true'])
java_class = Rjb::import('com.lowagie.text.pdf.PdfReader') 

ruby version: ruby-2.0.0-p451 change to ruby-1.9.3-p448 works.

Strip resque:

Kernel.fork { require 'rjb'; point = Rjb::import('java.awt.Point'); puts "haha"; Rjb::load('/Users/halida/data/workspace/sources/pdf-stamper/ext/iText-4.2.0.jar', ['-Djava.awt.headless=true']); puts "yes"; java_class = Rjb::import('com.lowagie.text.pdf.PdfReader'); puts 'end' }

In 2.0.0, it fails without any error, in 1.9.3, it tells me:

in `import': com/lowagie/text/pdf/PdfReader (NoClassDefFoundError)

[arton]

Hi do you use rvm, rbenv type of ruby environment changer ? If you do this, you should know about binary compatibility of gems. Because of Rjb is ruby extended library or native library, it should link with each ruby binary. So if you installed 3 version of ruby, you should 'gem install rjb' three times for each instance of ruby.

[halida]

According to above code, it works when call Rjb::import, and fails at Rjb::load, that confirm rjb installed.

[arton]

Loading gem only requires ruby's loading mechanism, not depending on the compilation time linking. However, if you met the failure of Rjb::load, it should be Rjb couldn't detect or load JVM. Your runtime environment does not point JVM by JAVA_HOME and LD_LIBRARY_PATH correctly.

[halida]

I will try to work this out..

[halida]

Hi, I've managed to create a demo for this bug:

https://github.com/halida/rjb_bug

clone it, bundle install, then make, it will run a function work both in main process and Kernel.fork, In Kernel.fork, it cannot import class.

[arton]

Hi Are you sure ? Why don't you believe you yourself and your environment is wrong. Why do you name your buggy environment as rjb-bug. Of course, Rjb has no bugs. If I was you, I named my_buggy_envionment_destroys_rjb instead of rjb_bug.

I am busy and have no responsibility for spending time for such wrong person, but kindly I have tested. --- the below is copy of my console arton@nomad:~/devl/github/rjb_bug$ make bundle exec ruby run.rb DL is deprecated, please use Fiddle java version: 1.7.0_21 rjb can load java native class rjb load iText jar rjb load iText class

java version: 1.7.0_21 rjb can load java native class rjb load iText jar rjb load iText class arton@nomad:~/devl/github/rjb_bug$

[arton]

halida, why do you lock rjb version 1.3.9 ? It's very old and I don't say nothing about that version except for (according to the above my console copy) it can work. Current Rjb version is 1.4.9 and it says nothing about DL.

[arton]

Also I found your run.rb has a bug. Because FILE points run.rb , it can't run if removing the first line. In other words, you can simply write java_work as: jversion = ... # rjb was already required in pdf/stamper puts ... point = ... puts ... java_class = ... #you already loaded it in pdf/stamper puts ...