Cannot expand ~

[chris-downs]

In redstorm v0.6.6 our topology dies when it hits include 'red_storm' due to the use of the tilde shortcut at this line of environment.rb:

50: DEFAULT_STORM_CONF_FILE = File.expand_path("~/.storm/storm.yaml").

Edit: To be clear, the topology submits fine from the local machine, the failure occurs inside of the JVM when storm initializes the JRuby spout and bolts.

2013-10-28 22:58:43 util [ERROR] Async loop died! java.lang.RuntimeException: org.jruby.exceptions.RaiseException: (ArgumentError) couldn't find HOME environment -- expanding ~' at redstorm.storm.jruby.JRubyBolt.newProxyBolt(JRubyBolt.java:85) at redstorm.storm.jruby.JRubyBolt.prepare(JRubyBolt.java:42) at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610) at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:679) Caused by: org.jruby.exceptions.RaiseException: (ArgumentError) couldn't find HOME environment -- expanding~' at org.jruby.RubyFile.expand_path(org/jruby/RubyFile.java:723)

I am able to fix the issue by using an absolute path there. I understand that that isn't the correct fix since $HOME can vary between environments. I'm sure this is a configuration issue on my end or else you would have seen it as well. I just don't know how to resolve it. I've tried setting ENV['HOME'] explicitly inside of the ruby files, as well as passing in -Duser.home to the nimbus\sup\workers.

We are using Storm v0.8.2

Java environment info: 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:host.name=lt2-supervisor-i-ecca338b 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.version=1.6.0_27 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.vendor=Sun Microsystems Inc. 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:java.compiler= 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.name=Linux 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.arch=amd64 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:os.version=3.2.0-48-virtual 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.name=storm 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.home=/home/storm 2013-10-28 21:50:32 ZooKeeper [INFO] Client environment:user.dir=/opt/storm/storm-0.8.2

I am unable to provide a full stack trace due to there being sensitive information in the file paths, but let me know if there is any other info I can give.

[colinsurprenant]

Thanks for the report. This is clearly an issue and this path in not even required upon topology initialization, it is only used for submission.

I guess a workaround would be to make sure there is a $HOME set in the environment running the Storm cluster. Can you verify this?

[airhorns]

@chris-downs did you end up figuring this one out?

[airhorns]

Turns out my problem was cause of runit (whoopth). Since runit descends from the init process it has no environment at all. Just had to export the HOME var before running the storm binary. We use chef to manage our runit services so here's what mine looks like:

#!/bin/sh
exec 2>&1
export HOME=/home/<%=@options[:user]%>
exec chpst -u <%=@options[:user]%> <%=@options[:install_dir]%>/bin/supervisor-control start

which when compiled looks something like

#!/bin/sh
exec 2>&1
export HOME=/home/storm
exec chpst -u storm /u/apps/storm/bin/supervisor-control start

[colinsurprenant]

eh! thanks for the followup! I will keep the issue open to remove the useless dependency on $HOME / ~ upon topology initialization.

[ismith]

I didn't notice this before I submitted #105, which should fix the bug; not sure how you want to handle combining these two issues, or just close one and merge the other.