--1.9 does not appear to be sent to cluster

[adsummos]

I have jruby set to 1.9 by default and I also included the --1.9 flag on redstorm cluster and when I check the JRUBY_OPTS, they are nil and 1.9 methods don't exist.

[colinsurprenant]

Weird. How did you set 1.9 by default? using export JRUBY_OPTS=--1.9 ? when you type ruby -v what do you get?

[adsummos]

I've tried exporting JRUBY_OPTS and compiling with 1.9 as the default, but no matter what I do the cluster defaults to 1.8. I can't figure out how you're telling Storm to use jruby-complete in 1.9 mode

ruby -v jruby 1.6.8 (ruby-1.9.2-p312) (2012-09-18 1772b40) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_35) [darwin-x86_64-java]

[adsummos]

I also seem to be having problems with rubygems, maybe the two issues are related somehow?

java.lang.RuntimeException: org.jruby.exceptions.RaiseException: (LoadError) no such file to load -- active_support/concern at redstorm.storm.jruby.JRubyTransactionalSpout.newProxySpout(JRubyTransactionalSpout.java:81) at redstorm.storm.jruby.JRubyTransactionalSpout.getEmitter(JRubyTransactionalSpout.java:52) at backtype.storm.transactional.TransactionalSpoutBatchExecutor.prepare(TransactionalSpoutBatchExecutor.java:31) at backtype.storm.coordination.CoordinatedBolt.prepare(CoordinatedBolt.java:196) at backtype.storm.daemon.executor$fn4036.invoke(executor.clj:531) at clojure.lang.MultiFn.invoke(MultiFn.java:167) at backtype.storm.daemon.executor$mk_executor$fn__3915.invoke(executor.clj:249) at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:248) at backtype.storm.daemon.worker$fn4324$exec_fn1207auto__4325$iter43304334$fn4335.invoke(worker.clj:353) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:60) at clojure.lang.Cons.next(Cons.java:39) at clojure.lang.RT.next(RT.java:587) at clojure.core$next.invoke(core.clj:64) at clojure.core$dorun.invoke(core.clj:2726) at clojure.core$doall.invoke(core.clj:2741) at backtype.storm.daemon.worker$fn4324$exec_fn1207auto__4325.invoke(worker.clj:353) at clojure.lang.AFn.applyToHelper(AFn.java:185) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:601) at backtype.storm.daemon.worker$fn4324$mk_worker4380.doInvoke(worker.clj:322) at clojure.lang.RestFn.invoke(RestFn.java:512) at backtype.storm.daemon.worker$_main.invoke(worker.clj:432) at clojure.lang.AFn.applyToHelper(AFn.java:172) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.daemon.worker.main(Unknown Source)

[adsummos]

also, everything works fine in local mode

[colinsurprenant]

for the 1.9 mode, the -Djruby.compat.version=TOKEN is passed to either the java command for local mode or the storm command for cluster mode.

if no --1.8 or --1.9 is passed, the TOKEN will be set to RUBY1_8 or RUBY1_9 depending on the current Ruby environment version. Otherwise it is set using the passed switch.

Can you check if -Djruby.compat.version= is correctly passed in local and cluster mode?

Are you using the 0.6.4?

[colinsurprenant]

for the gem, did you add the gem in a Gemfile under a group, lets say "mygroup" and run: $ bundle install $ redstorm bundle mygroup

Also, it might very well be related to your 1.8/1.9 issue since bundler uses the Ruby version in its gem paths so if a gem is installed in 1.9 and the topology runs in 1.8, it won't load the gem.

[colinsurprenant]

to make sure you are using 1.9 everywhere, start from scratch (remove your gemset if using bundler) and try to always force jruby in 1.9 using jruby --1.9 -S bundle install and jruby --1.9 -S redstorm cluster --1.9 etc

personally, I install JRuby with rvm in its default mode (1.8) and use export JRUBY_OPTS=--1.9 and 1.9 mode is always correctly carried.

[adsummos]

the gems are in the default group and I did a restorm bundle default. Could that be the gem issue?

[colinsurprenant]

when you did the restorm bundle default did you see the message installing gem xxx into yyy ? also if you look into target/gems/gems/ is the gem present?

[colinsurprenant]

what redstorm version are you using? issue #49 closed in 0.6.3 was about a problem handling the bundler default group.

[adsummos]

yes, and yes and I am using 0.6.4. I tried starting from scratch with jruby --1.9 -S and got the same result. I briefly tried with jruby 1.7, but got some strange yaml error and couldn't even deploy the jar

[adsummos]

I am currently using jruby 1.6.8 default rvm install

[colinsurprenant]

Are you still having the 1.9 issue?

[adsummos]

yes

[colinsurprenant]

Hm. can you create a minimal "project" (Gemfile, topology) which can reproduce this problem? I will set it up locally and try to reproduce the problem here.

[adsummos]

ok, I'll put something together first thing in the morning

[colinsurprenant]

in the meantime could you copy/gist/email me your Gemfile?

[adsummos]

source :rubygems

gem "redstorm", :git => "git@github.com:adsummos/redstorm.git", :require => 'red_storm'
# do not require /all to save on overhead of loading these entire libraries
gem 'activesupport', '~> 3.2.0', :require => 'active_support'
gem 'activemodel', '~> 3.2.0', :require => 'active_model'
gem 'activerecord', '~> 3.2.0', :require => 'active_record'
gem 'activerecord-jdbc-adapter'
gem 'activerecord-jdbcpostgresql-adapter'
gem 'jdbc-postgres'
gem 'rake', "~> 0.9"
gem 'aws-sdk'
gem 'jruby-openssl' # this seems to be required for jruby
gem 'retryable'
gem 'redis'
gem 'redis-namespace'
gem 'airbrake'

gem 'json-jruby'

# mongo
gem 'mongo'
gem 'bson'

[adsummos]

I get: java.lang.RuntimeException: org.jruby.exceptions.RaiseException: (NoMethodError) undefined method require_relative' for main:Object at redstorm.storm.jruby.JRubyTransactionalSpout.newProxySpout(JRubyTransactionalSpout.java:81) at redstorm.storm.jruby.JRubyTransactionalSpout.getComponentConfiguration(JRubyTransactionalSpout.java:71) at backtype.storm.transactional.TransactionalSpoutCoordinator.open(TransactionalSpoutCoordinator.java:57) at backtype.storm.daemon.executor$fn__3976.invoke(executor.clj:415) at clojure.lang.MultiFn.invoke(MultiFn.java:167) at backtype.storm.daemon.executor$mk_executor$fn__3915.invoke(executor.clj:249) at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:248) at backtype.storm.daemon.worker$fn__4324$exec_fn__1207__auto____4325$iter__4330__4334$fn__4335.invoke(worker.clj:353) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:60) at clojure.lang.RT.seq(RT.java:473) at clojure.core$seq.invoke(core.clj:133) at clojure.core$dorun.invoke(core.clj:2725) at clojure.core$doall.invoke(core.clj:2741) at backtype.storm.daemon.worker$fn__4324$exec_fn__1207__auto____4325.invoke(worker.clj:353) at clojure.lang.AFn.applyToHelper(AFn.java:185) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:601) at backtype.storm.daemon.worker$fn__4324$mk_worker__4380.doInvoke(worker.clj:322) at clojure.lang.RestFn.invoke(RestFn.java:512) at backtype.storm.daemon.worker$_main.invoke(worker.clj:432) at clojure.lang.AFn.applyToHelper(AFn.java:172) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.daemon.worker.main(Unknown Source) Caused by: org.jruby.exceptions.RaiseException: (NoMethodError) undefined methodrequire_relative' for main:Object

when deploying https://github.com/adsummos/storm-test

[adsummos]

sorry, wrong stacktrace

java.lang.RuntimeException: org.jruby.exceptions.RaiseException: (NoMethodError) undefined method require_relative' for main:Object at redstorm.storm.jruby.JRubySpout.newProxySpout(JRubySpout.java:104) at redstorm.storm.jruby.JRubySpout.open(JRubySpout.java:42) at backtype.storm.daemon.executor$fn__3976.invoke(executor.clj:415) at clojure.lang.MultiFn.invoke(MultiFn.java:167) at backtype.storm.daemon.executor$mk_executor$fn__3915.invoke(executor.clj:249) at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:248) at backtype.storm.daemon.worker$fn__4324$exec_fn__1207__auto____4325$iter__4330__4334$fn__4335.invoke(worker.clj:353) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:60) at clojure.lang.Cons.next(Cons.java:39) at clojure.lang.LazySeq.next(LazySeq.java:92) at clojure.lang.RT.next(RT.java:587) at clojure.core$next.invoke(core.clj:64) at clojure.core$dorun.invoke(core.clj:2726) at clojure.core$doall.invoke(core.clj:2741) at backtype.storm.daemon.worker$fn__4324$exec_fn__1207__auto____4325.invoke(worker.clj:353) at clojure.lang.AFn.applyToHelper(AFn.java:185) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:601) at backtype.storm.daemon.worker$fn__4324$mk_worker__4380.doInvoke(worker.clj:322) at clojure.lang.RestFn.invoke(RestFn.java:512) at backtype.storm.daemon.worker$_main.invoke(worker.clj:432) at clojure.lang.AFn.applyToHelper(AFn.java:172) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.daemon.worker.main(Unknown Source) Caused by: org.jruby.exceptions.RaiseException: (NoMethodError) undefined methodrequire_relative' for main:Object

[adsummos]

is there possibly something different on your cluster that is allowing this to work? We are currently using a default cluster deployed by storm-deploy

[adsummos]

I fixed the 1.9 issue with worker.childopts: "-Xmx768m -Djruby.compat.version=RUBY1_9" I'm going to work on the gem issue now

[adsummos]

the gem file issue was fixed by adding a require 'red_storm' in the topology.rb

[colinsurprenant]

ah crap, should have checked this first thing for the gem.

for the 1.9 issue, I guess I will have to try this with storm-deploy, I don't see why it is not carried.

in simple_topology.rb I set the worker.childopts using

# set the JRuby compatibility mode option for Storm workers, default to current JRuby mode
defaults = {"topology.worker.childopts" => "-Djruby.compat.version=#{RedStorm.jruby_mode_token}"}

can you try forcing it in the topology configure block, like this:

configure do |env|
  debug true
  set "topology.worker.childopts", "-Djruby.compat.version=RUBY1_9"
end

[adsummos]

I think that's the issue. I'm just using the proxy classes, not the DSL, so I didn't get that set by default. That's the piece I was missing to understand how storm knew to run it in 1.9 mode.

On Nov 2, 2012, at 12:30 PM, Colin Surprenant wrote:

ah crap, should have checked this first thing for the gem.

for the 1.9 issue, I guess I will have to try this with storm-deploy, I don't see why it is not carried.

in simple_topology.rb I set the worker.childopts using

set the JRuby compatibility mode option for Storm workers, default to current JRuby mode

defaults = {"topology.worker.childopts" => "-Djruby.compat.version=#{RedStorm.jruby_mode_token}"}

can you try forcing it in the topology configure block, like this:

configure do |env| debug true set "topology.worker.childopts", "-Djruby.compat.version=RUBY1_9" end

— Reply to this email directly or view it on GitHubhttps://github.com/colinsurprenant/redstorm/issues/53#issuecomment-10020161.

[colinsurprenant]

AH! bingo. I guess we've resolved all issues now? I'll go ahead and close this, if there is anything else please don't hesitate to reopen or submit other issues.

I will create a ticket to not forget to document this use-case.

Thanks for your patience.