search

pystorm / streamparse

Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.
http://streamparse.readthedocs.io/
Apache License 2.0
1.49k stars 218 forks source link

run locally streamparse Error #85

Closed hadisotudeh closed 9 years ago

hadisotudeh commented 10 years ago

[hadi@hadi STORM]$ sparse quickstart word

Creating your word streamparse project... ... ... ...

then I ran this command: cd word sparse run

...

but the result is :

6509 [Thread-9-count-bolt] INFO backtype.storm.util - Halting process: ("Worker died") Traceback (most recent call last): File "/usr/bin/sparse", line 9, in load_entry_point('streamparse==1.0.1', 'console_scripts', 'sparse')() File "/usr/lib/python2.7/site-packages/streamparse/cmdln.py", line 82, in main run_local_topology(args["--name"], time, par, options, args["--debug"]) File "/usr/lib/python2.7/site-packages/invoke/tasks.py", line 111, in call result = self.body(_args, *_kwargs) File "/usr/lib/python2.7/site-packages/streamparse/ext/invoke.py", line 170, in run_local_topology run(full_cmd) File "/usr/lib/python2.7/site-packages/invoke/runner.py", line 160, in run raise Failure(result) invoke.exceptions.Failure: Command execution failure!

What should I do?

amontalenti commented 10 years ago

@hadisotudeh Can you run sparse run -t 120? That will run the Storm LocalCluster for 120 seconds. By default, the topology only runs for 5 seconds and then halts.

hadisotudeh commented 10 years ago

@amontalenti I tried that and the result is :

sparse run -d -t 120

Running wordcount topology... Routing Python logging to /home/hadi/STORM/wordCount/logs. Running lein command to run local cluster: lein run -m streamparse.commands.run/-main topologies/wordcount.clj -t 120 --debug --option 'topology.workers=2' --option 'topology.acker.executors=2' --option 'streamparse.log.path="/home/hadi/STORM/wordCount/logs"' --option 'streamparse.log.level="debug"' 2718 [main] INFO backtype.storm.zookeeper - Starting inprocess zookeeper at port 2000 and dir /tmp/7c6f44b2-5b0e-4f25-b43b-87ee1063664d 2898 [main] INFO backtype.storm.daemon.nimbus - Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/37537372-3a72-492c-88d6-face67d837e9", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.max.task.parallelism" nil, "storm.messaging.netty.transfer.batch.size" 262144} 2919 [main] INFO backtype.storm.daemon.nimbus - Using default scheduler 2987 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 3058 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 3063 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 3198 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 3198 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 3205 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 4271 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14a10489cd90000, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [zookeeper-3.4.5.jar:3.4.5-1392090] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20] 4274 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 4289 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 4289 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 4331 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 4335 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 4335 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 4337 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 5346 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 5351 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 5353 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 5354 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 5363 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 5365 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 5458 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 5458 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 5469 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 5470 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 5470 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 5478 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 5481 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 5482 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 5585 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 5585 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 5609 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/869b73bb-c13d-46ba-94e8-581b56957146", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1024 1025 1026), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.max.task.parallelism" nil, "storm.messaging.netty.transfer.batch.size" 262144} 6868 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 6883 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 6883 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 6883 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 6892 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 6903 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 6904 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 7007 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 7020 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 7047 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor with id 7cf23a45-c01b-48d2-9713-e096bcbb5db0 at host hadi 7050 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/f7ac4b20-ef2f-4c85-8248-b3bddb369c3e", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1027 1028 1029), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.max.task.parallelism" nil, "storm.messaging.netty.transfer.batch.size" 262144} 7057 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 7060 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 7060 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 7061 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 8067 [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 8071 [main-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 8072 [main-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 8177 [main-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 8177 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 9197 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor with id 54702e40-5a00-481e-8274-3a76d8fb2ef3 at host hadi 9293 [main] INFO backtype.storm.daemon.nimbus - Received topology submission for wordcount with conf {"storm.id" "wordcount-1-1417612408", "topology.acker.executors" 2, "streamparse.log.path" "/home/hadi/STORM/wordCount/logs", "topology.kryo.decorators" (), "topology.name" "wordcount", "topology.max.spout.pending" 5000, "topology.debug" true, "topology.kryo.register" nil, "topology.message.timeout.secs" 60, "topology.workers" 2, "topology.max.task.parallelism" nil, "streamparse.log.level" "debug"} 9325 [main] INFO backtype.storm.daemon.nimbus - Activating wordcount: wordcount-1-1417612408 9439 [main] INFO backtype.storm.scheduler.EvenScheduler - Available slots: (["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1025] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1026] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1027] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1028] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1029]) 9473 [main] INFO backtype.storm.daemon.nimbus - Setting new assignment for topology id wordcount-1-1417612408: #backtype.storm.daemon.common.Assignment{:master-code-dir "/tmp/37537372-3a72-492c-88d6-face67d837e9/nimbus/stormdist/wordcount-1-1417612408", :node->host {"54702e40-5a00-481e-8274-3a76d8fb2ef3" "hadi", "7cf23a45-c01b-48d2-9713-e096bcbb5db0" "hadi"}, :executor->node+port {[2 2] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1027], [3 3] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024], [4 4] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1027], [5 5] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024], [6 6] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1027], [7 7] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024], [8 8] ["54702e40-5a00-481e-8274-3a76d8fb2ef3" 1027], [9 9] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024], [1 1] ["7cf23a45-c01b-48d2-9713-e096bcbb5db0" 1024]}, :executor->start-time-secs {[1 1] 1417612408, [9 9] 1417612408, [8 8] 1417612408, [7 7] 1417612408, [6 6] 1417612408, [5 5] 1417612408, [4 4] 1417612408, [3 3] 1417612408, [2 2] 1417612408}} 9501 [Thread-3] INFO backtype.storm.daemon.supervisor - Downloading code for storm id wordcount-1-1417612408 from /tmp/37537372-3a72-492c-88d6-face67d837e9/nimbus/stormdist/wordcount-1-1417612408 9812 [Thread-3] INFO backtype.storm.daemon.supervisor - Copying resources at file:/home/hadi/STORM/wordCount/_resources/resources to /tmp/869b73bb-c13d-46ba-94e8-581b56957146/supervisor/stormdist/wordcount-1-1417612408/resources 9815 [Thread-3] INFO backtype.storm.daemon.supervisor - Finished downloading code for storm id wordcount-1-1417612408 from /tmp/37537372-3a72-492c-88d6-face67d837e9/nimbus/stormdist/wordcount-1-1417612408 9848 [Thread-4] INFO backtype.storm.daemon.supervisor - Launching worker with assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id "wordcount-1-1417612408", :executors ([3 3] [5 5] [7 7] [9 9] [1 1])} for this supervisor 7cf23a45-c01b-48d2-9713-e096bcbb5db0 on port 1024 with id d2c7759d-a691-4ece-a11e-906695cf8b5d 9853 [Thread-4] INFO backtype.storm.daemon.worker - Launching worker for wordcount-1-1417612408 on 7cf23a45-c01b-48d2-9713-e096bcbb5db0:1024 with id d2c7759d-a691-4ece-a11e-906695cf8b5d and conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/869b73bb-c13d-46ba-94e8-581b56957146", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1024 1025 1026), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.max.task.parallelism" nil, "storm.messaging.netty.transfer.batch.size" 262144} 9857 [Thread-4] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 9873 [Thread-4-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 9873 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 9873 [Thread-4-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 10068 [Thread-5] INFO backtype.storm.daemon.supervisor - Downloading code for storm id wordcount-1-1417612408 from /tmp/37537372-3a72-492c-88d6-face67d837e9/nimbus/stormdist/wordcount-1-1417612408 10219 [Thread-5] INFO backtype.storm.daemon.supervisor - Copying resources at file:/home/hadi/STORM/wordCount/_resources/resources to /tmp/f7ac4b20-ef2f-4c85-8248-b3bddb369c3e/supervisor/stormdist/wordcount-1-1417612408/resources 10220 [Thread-5] INFO backtype.storm.daemon.supervisor - Finished downloading code for storm id wordcount-1-1417612408 from /tmp/37537372-3a72-492c-88d6-face67d837e9/nimbus/stormdist/wordcount-1-1417612408 10229 [Thread-6] INFO backtype.storm.daemon.supervisor - Launching worker with assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id "wordcount-1-1417612408", :executors ([2 2] [4 4] [6 6] [8 8])} for this supervisor 54702e40-5a00-481e-8274-3a76d8fb2ef3 on port 1027 with id d7179670-71e1-4f73-81c5-10cdf84a7b44 10231 [Thread-6] INFO backtype.storm.daemon.worker - Launching worker for wordcount-1-1417612408 on 54702e40-5a00-481e-8274-3a76d8fb2ef3:1027 with id d7179670-71e1-4f73-81c5-10cdf84a7b44 and conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "storm.messaging.netty.flush.check.interval.ms" 10, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/f7ac4b20-ef2f-4c85-8248-b3bddb369c3e", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1027 1028 1029), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.max.task.parallelism" nil, "storm.messaging.netty.transfer.batch.size" 262144} 10231 [Thread-6] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 10234 [Thread-6-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 10234 [Thread-6-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 10337 [Thread-6-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 10337 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 10337 [Thread-6-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none 10347 [Thread-6] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 10353 [Thread-6-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 10353 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 10881 [Thread-4] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 10886 [Thread-4-SendThread(localhost.localdomain:2000)] ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost.localdomain/0:0:0:0:0:0:0:1:2000 10887 [Thread-4-SendThread(localhost.localdomain:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.8.0_20] at sun.nio.ch.Net.connect(Net.java:449) ~[na:1.8.0_20] at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_20] at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958) ~[zookeeper-3.4.5.jar:3.4.5-1392090] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) ~[zookeeper-3.4.5.jar:3.4.5-1392090] 10991 [Thread-4-EventThread] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 10992 [ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered. 11370 [Thread-4] INFO backtype.storm.daemon.executor - Loading executor sentence-spout-1:[3 3] 11382 [Thread-4] INFO backtype.storm.daemon.task - Emitting: sentence-spout-1 system ["startup"] 11383 [Thread-4] INFO backtype.storm.daemon.executor - Loaded executor tasks sentence-spout-1:[3 3] 11411 [Thread-4] INFO backtype.storm.daemon.executor - Finished loading executor sentence-spout-1:[3 3] 11417 [Thread-8-sentence-spout-1] INFO backtype.storm.daemon.executor - Opening spout sentence-spout-1:(3) 11419 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor acker:[2 2] 11421 [Thread-6] INFO backtype.storm.daemon.task - Emitting: acker system ["startup"] 11422 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks acker:[2 2] 11436 [Thread-4] INFO backtype.storm.daemon.executor - Loading executor sentence-spout-2:[5 5] 11440 [Thread-8-sentence-spout-1] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11440 [Thread-6] INFO backtype.storm.daemon.executor - Timeouts disabled for executor acker:[2 2] 11442 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading executor acker:[2 2] 11443 [Thread-4] INFO backtype.storm.daemon.task - Emitting: sentence-spout-2 system ["startup"] 11443 [Thread-4] INFO backtype.storm.daemon.executor - Loaded executor tasks sentence-spout-2:[5 5] 11441 [Thread-10-acker] INFO backtype.storm.daemon.executor - Preparing bolt acker:(2) 11449 [Thread-10-acker] INFO backtype.storm.daemon.executor - Prepared bolt acker:(2) 11462 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor sentence-spout-1:[4 4] 11450 [Thread-4] INFO backtype.storm.daemon.executor - Finished loading executor sentence-spout-2:[5 5] 11465 [Thread-12-sentence-spout-2] INFO backtype.storm.daemon.executor - Opening spout sentence-spout-2:(5) 11467 [Thread-6] INFO backtype.storm.daemon.task - Emitting: sentence-spout-1 system ["startup"] 11467 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks sentence-spout-1:[4 4] 11475 [Thread-4] INFO backtype.storm.daemon.executor - Loading executor sentenceSplitter2:[7 7] 11478 [Thread-4] INFO backtype.storm.daemon.task - Emitting: sentenceSplitter2 system ["startup"] 11478 [Thread-4] INFO backtype.storm.daemon.executor - Loaded executor tasks sentenceSplitter2:[7 7] 11484 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading executor sentence-spout-1:[4 4] 11491 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor sentenceSplitter1:[6 6] 11484 [Thread-15-sentence-spout-1] INFO backtype.storm.daemon.executor - Opening spout sentence-spout-1:(4) 11494 [Thread-15-sentence-spout-1] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11494 [Thread-12-sentence-spout-2] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11497 [Thread-6] INFO backtype.storm.daemon.task - Emitting: sentenceSplitter1 system ["startup"] 11497 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks sentenceSplitter1:[6 6] 11504 [Thread-4] INFO backtype.storm.daemon.executor - Finished loading executor sentenceSplitter2:[7 7] 11505 [Thread-17-sentenceSplitter2] INFO backtype.storm.daemon.executor - Preparing bolt sentenceSplitter2:(7) 11505 [Thread-17-sentenceSplitter2] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11512 [Thread-4] INFO backtype.storm.daemon.executor - Loading executor word-counter:[9 9] 11514 [Thread-4] INFO backtype.storm.daemon.task - Emitting: word-counter system ["startup"] 11514 [Thread-4] INFO backtype.storm.daemon.executor - Loaded executor tasks word-counter:[9 9] 11519 [Thread-8-sentence-spout-1] ERROR backtype.storm.util - Async loop died! java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: /usr/bin/python: Error while finding spec for 'streamparse.run' (<class 'ImportError'>: No module named 'streamparse')

at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:68) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.spout.ShellSpout.open(ShellSpout.java:54) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.executor$fn__3284$fn__3299.invoke(executor.clj:520) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]

11519 [Thread-8-sentence-spout-1] ERROR backtype.storm.daemon.executor - java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: /usr/bin/python: Error while finding spec for 'streamparse.run' (<class 'ImportError'>: No module named 'streamparse')

at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:68) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.spout.ShellSpout.open(ShellSpout.java:54) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.executor$fn__3284$fn__3299.invoke(executor.clj:520) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]

11534 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading executor sentenceSplitter1:[6 6] 11534 [Thread-18-sentenceSplitter1] INFO backtype.storm.daemon.executor - Preparing bolt sentenceSplitter1:(6) 11535 [Thread-18-sentenceSplitter1] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11563 [Thread-4] INFO backtype.storm.daemon.executor - Finished loading executor word-counter:[9 9] 11564 [Thread-20-word-counter] INFO backtype.storm.daemon.executor - Preparing bolt word-counter:(9) 11564 [Thread-20-word-counter] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 11584 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor word-counter:[8 8] 11593 [Thread-17-sentenceSplitter2] ERROR backtype.storm.util - Async loop died! java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: /usr/bin/python: Error while finding spec for 'streamparse.run' (<class 'ImportError'>: No module named 'streamparse')

at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:68) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.executor$fn__3352$fn__3364.invoke(executor.clj:690) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]

11593 [Thread-17-sentenceSplitter2] ERROR backtype.storm.daemon.executor - java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: /usr/bin/python: Error while finding spec for 'streamparse.run' (<class 'ImportError'>: No module named 'streamparse')

at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:68) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.executor$fn__3352$fn__3364.invoke(executor.clj:690) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]

11594 [Thread-8-sentence-spout-1] INFO backtype.storm.util - Halting process: ("Worker died") 11597 [Thread-6] INFO backtype.storm.daemon.task - Emitting: word-counter system ["startup"] Traceback (most recent call last): File "/usr/bin/sparse", line 9, in load_entry_point('streamparse==1.0.1', 'console_scripts', 'sparse')() File "/usr/lib/python2.7/site-packages/streamparse/cmdln.py", line 82, in main run_local_topology(args["--name"], time, par, options, args["--debug"]) File "/usr/lib/python2.7/site-packages/invoke/tasks.py", line 111, in __call result = self.body(_args, *_kwargs) File "/usr/lib/python2.7/site-packages/streamparse/ext/invoke.py", line 174, in run_local_topology run(full_cmd) File "/usr/lib/python2.7/site-packages/invoke/runner.py", line 160, in run raise Failure(result) invoke.exceptions.Failure: Command execution failure!

Exit code: 1

Stderr:

What is your suggestion?

wilsoncao commented 9 years ago

I have the same problem with you, any idea to solve it?

amontalenti commented 9 years ago

This part of your log output:

/usr/bin/python: Error while finding spec for 'streamparse.run'

Indicates that when python -m streamparse.run is attempting to be run, it is not succeeding because 'streamparse' is not available in your PYTHONPATH. The reason for this is probably that you don't have your virtualenv that includes a streamparse installation properly activated, I think. Closing.