Error while loading shared libraries

[johanneshk]

Hey,

I am currently trying to move from pyleus local to an actual storm cluster. Therefore I run storm nimbus, ui, supervisor and zookeeper on the same machine.

Right now I want to submit the word_count example to the cluster. But a worker log shows the following error:

2015-04-10 14:52:21 b.s.d.executor [INFO] Preparing bolt log-results:(6)
2015-04-10 14:52:21 b.s.u.ShellProcess [INFO] Storm multilang serializer: com.yelp.pyleus.serializer.MessagePackSerializer
2015-04-10 14:52:21 b.s.util [ERROR] Async loop died!
java.lang.RuntimeException: Error when launching multilang subprocess
pyleus_venv/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory

        at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:64) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.executor$fn__5641$fn__5653.invoke(executor.clj:690) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.util$async_loop$fn__457.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.io.IOException: Stream closed
        at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:434) ~[na:1.7.0_75]
        at java.io.OutputStream.write(OutputStream.java:116) ~[na:1.7.0_75]
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.7.0_75]
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.7.0_75]
        at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.7.0_75]
        at com.yelp.pyleus.serializer.MessagePackSerializer.writeMessage(MessagePackSerializer.java:208) ~[stormjar.jar:na]
        at com.yelp.pyleus.serializer.MessagePackSerializer.connect(MessagePackSerializer.java:65) ~[stormjar.jar:na]
        at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:62) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        ... 5 common frames omitted

Apparently "libpython2.7.so.1.0" cannot be found. How can I fix this? local mode works like charm.

Thanks! Johannes

[johanneshk]

I could narrow down the problem a bit: It seems like when submitting a topology to the cluster, some things are missing: When I start the topology in local mode I have to 'module load oraclejdk python' to load the according modules. I think this does not happen when submitting to the cluster. Can I somewhere tell the program to also load these modules? When I login on the cluster, the modules are automatically loaded (have it in bashrc / zshrc). So what I need is someplace to run some commands before a topology is started.

[johanneshk]

To answer my own question: My findings are:

• bash is launched as non-login, non-interactive shell for the shell bolt
• in this mode bash looks for a path/file in the environment variable 'BASH_ENV', if it exists that file is sourced
• now I only had to create a file that loads the module and point the env. var. to the location of the file

That's it. Now it works.