barrywhart
commented
11 years ago I think this is the main problem:
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement thrift==0.8.0
No distributions at all found for thrift==0.8.0From the first line, it appears that the worker cannot access pypi.python.org. Do you know why this would happen?
Also, you say you have Thrift 0.9.2 installed, but Petrel is trying to install 0.8.0. Note that in setup.py, Petrel requests Thrift 0.8.0:
,install_requires=[
'simplejson==2.6.1',
# Request specific Thrift version. Storm is in Java and may be sensitive to version incompatibilities.
'thrift==0.8.0',
'PyYAML==3.10',
]which is why that's the version it's trying to install.
w7374520
1、Version information： storm 0.8.2 python 2.7.5 thrift 0.9.2 centos 5U5
2、Use Petrel to do the test，From the site: https://github.com/AirSage/Petrel
3、Log the error message, can you help me analyze error? Thank you.
[root@snamenode resources]# pwd /data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources [root@snamenode resources]# less petrel7768_wordcount.log
Beginning task setup
Using existing venv: /tmp/petrel-wordcount-1-1369714538/venv Task setup took 47 seconds Launching: python -m petrel.run wordcount /data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/petrel7768_wordcount.log /usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/run.py invoked with the following arguments: ['wordcount', '/data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/petrel7768_wordcount.log'] python version: 2.7.5 user=root PATH=/tmp/petrel-wordcount-1-1369714538/venv/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/jdk1.6.0_41/bin:/usr/java/jre/bin:/usr/local/hadoop/bin:/usr/local/hadoop/hive/bin:/usr/local/ant/bin:/usr/local/php5/bin:/usr/local/scribe/bin:/usr/local/hbase/bin:/usr/local/maven/bin:/data/local/storm/bin:/usr/local/thrift/bin:/root/bin LD_LIBRARY_PATH=/usr/local/jdk1.6.0_41/jre/lib/amd64/server:/usr/local/jdk1.6.0_41/jre/lib/amd64:/usr/local/jdk1.6.0_41/jre/../lib/amd64:/usr/local/lib:/opt/local/lib:/usr/lib PYTHON_EGG_CACHE=/tmp/petrel-wordcount-1-1369714538/egg_cache [2013-05-28 12:16:34,675][storm][INFO]Tuple profiling enabled. Will log tuple processing times. [2013-05-28 12:16:34,677][storm][INFO]Task received setupInfo from Storm: {'pidDir': '/data/local/storm-local/workers/e7240da6-6692-4191-87e1-f1b43e90d950/pids', 'conf': {'supervisor.heartbeat.frequency.secs': 5, 'topology.stats.sample.rate': 0.05, 'topology.fall.back.on.java.serialization': True, 'topology.worker.childopts': '-Xmx128m', 'storm.id': 'wordcount-1-1369714538', 'drpc.worker.threads': 64, 'storm.zookeeper.servers': ['zookeeper'], 'transactional.zookeeper.root': '/transactional', 'supervisor.childopts': '-Xmx256m', 'storm.zookeeper.connection.timeout': 15000, 'zmq.threads': 1, 'topology.executor.send.buffer.size': 1024, 'petrel.host': 'snamenode', 'topology.enable.message.timeouts': True, 'storm.local.mode.zmq': False, 'nimbus.thrift.port': 6627, 'supervisor.worker.timeout.secs': 150, 'nimbus.cleanup.inbox.freq.secs': 600, 'nimbus.topology.validator': 'backtype.storm.nimbus.DefaultTopologyValidator', 'storm.local.dir': '/data/local/storm-local', 'nimbus.supervisor.timeout.secs': 60, 'topology.disruptor.wait.strategy': 'com.lmax.disruptor.BlockingWaitStrategy', 'worker.heartbeat.frequency.secs': 1, 'nimbus.inbox.jar.expiration.secs': 3600, 'supervisor.monitor.frequency.secs': 3, 'drpc.port': 3772, 'topology.trident.batch.emit.interval.millis': 500, 'topology.kryo.factory': 'backtype.storm.serialization.DefaultKryoFactory', 'storm.zookeeper.port': 2181, 'nimbus.childopts': '-Xmx512m', 'topology.workers': 1, 'topology.acker.tasks': None, 'topology.tick.tuple.freq.secs': None, 'supervisor.worker.start.timeout.secs': 120, 'supervisor.enable': True, 'nimbus.task.launch.secs': 120, 'topology.sleep.spout.wait.strategy.time.ms': 1, 'worker.childopts': '-Xmx128m', 'task.refresh.poll.secs': 10, 'topology.debug': False, 'nimbus.task.timeout.secs': 30, 'topology.message.timeout.secs': 150, 'nimbus.reassign': True, 'petrel.user': 'root', 'drpc.queue.size': 128, 'topology.ackers': 1, 'topology.state.synchronization.timeout.secs': 60, 'topology.executor.receive.buffer.size': 1024, 'nimbus.host': '10.14.21.227', 'ui.port': 28080, 'topology.name': 'wordcount', 'transactional.zookeeper.servers': None, 'drpc.invocations.port': 3773, 'nimbus.file.copy.expiration.secs': 600, 'topology.skip.missing.kryo.registrations': False, 'topology.acker.executors': 1, 'ui.childopts': '-Xmx768m', 'nimbus.monitor.freq.secs': 10, 'topology.max.task.parallelism': None, 'storm.cluster.mode': 'distributed', 'zmq.hwm': 0, 'dev.zookeeper.path': '/data/local/dev-storm-zookeeper', 'drpc.request.timeout.secs': 600, 'zmq.linger.millis': 5000, 'topology.error.throttle.interval.secs': 10, 'topology.worker.shared.thread.pool.size': 4, 'java.library.path': '/usr/local/lib:/opt/local/lib:/usr/lib', 'storm.zookeeper.session.timeout': 20000, 'topology.kryo.register': None, 'storm.zookeeper.retry.times': 5, 'topology.tasks': None, 'storm.zookeeper.root': '/storm', 'topology.spout.wait.strategy': 'backtype.storm.spout.SleepSpoutWaitStrategy', 'topology.max.error.report.per.interval': 5, 'task.heartbeat.frequency.secs': 3, 'supervisor.slots.ports': [6700, 6701, 6702, 6703], 'topology.transfer.buffer.size': 1024, 'topology.max.spout.pending': 1, 'topology.optimize': True, 'topology.kryo.decorators': [], 'storm.zookeeper.retry.interval': 1000, 'transactional.zookeeper.port': None, 'topology.receiver.buffer.size': 8}, 'context': {'task->component': {'1': '__acker', '3': 'split', '2': 'count', '4': 'spout'}}} [2013-05-28 12:16:34,677][storm][ERROR][Errno 32] Broken pipe Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 110, in sendMsgToParent old_stdout.flush() IOError: [Errno 32] Broken pipe [2013-05-28 12:16:34,678][petrel.run][ERROR]E_RUNFAILED_wordcount_snamenode_7768_StormIPCException [2013-05-28 12:16:34,678][storm][ERROR]Sent failure message ("E_RUNFAILED_wordcount_snamenode_7768_StormIPCException") to Storm Exception occurred in wordcount. Worker exiting. Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/run.py", line 100, in main getattr(module, 'run')() File "/data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/wordcount.py", line 49, in run WordCountBolt().run() File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 385, in run conf, context = initComponent() File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 271, in initComponent sendpid(setupInfo['pidDir']) File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 143, in sendpid sendMsgToParent({'pid':pid}) File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 116, in sendMsgToParent str(e))) StormIPCException: IOError error [Errno 32] in sendMsgToParent: [Errno 32] Broken pipe
[root@snamenode resources]# less petrel7775_virtualenv.log New python executable in /tmp/petrel-wordcount-1-1369714538/venv/bin/python2.7 Also creating executable in /tmp/petrel-wordcount-1-1369714538/venv/bin/python Installing setuptools............done. Installing pip...............done. Running virtualenv with interpreter /usr/local/bin/python2.7 Installing simplejson==2.6.1 Requirement already satisfied (use --upgrade to upgrade): simplejson==2.6.1 in /usr/local/lib/python2.7/site-packages Cleaning up... Installing thrift==0.8.0 Downloading/unpacking thrift==0.8.0 Cannot fetch index base URL https://pypi.python.org/simple/ Could not find any downloads that satisfy the requirement thrift==0.8.0 No distributions at all found for thrift==0.8.0 Storing complete log in /root/.pip/pip.log
[root@snamenode resources]# cat petrel7768_wordcount.log Beginning task setup
Using existing venv: /tmp/petrel-wordcount-1-1369714538/venv Task setup took 47 seconds Launching: python -m petrel.run randomsentence /data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/petrel7790_randomsentence.log /usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/run.py invoked with the following arguments: ['randomsentence', '/data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/petrel7790_randomsentence.log'] python version: 2.7.5 user=root PATH=/tmp/petrel-wordcount-1-1369714538/venv/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/jdk1.6.0_41/bin:/usr/java/jre/bin:/usr/local/hadoop/bin:/usr/local/hadoop/hive/bin:/usr/local/ant/bin:/usr/local/php5/bin:/usr/local/scribe/bin:/usr/local/hbase/bin:/usr/local/maven/bin:/data/local/storm/bin:/usr/local/thrift/bin:/root/bin LD_LIBRARY_PATH=/usr/local/jdk1.6.0_41/jre/lib/amd64/server:/usr/local/jdk1.6.0_41/jre/lib/amd64:/usr/local/jdk1.6.0_41/jre/../lib/amd64:/usr/local/lib:/opt/local/lib:/usr/lib PYTHON_EGG_CACHE=/tmp/petrel-wordcount-1-1369714538/egg_cache [2013-05-28 12:16:34,675][storm][INFO]Tuple profiling enabled. Will log tuple processing times. [2013-05-28 12:16:34,679][storm][INFO]Task received setupInfo from Storm: {'pidDir': '/data/local/storm-local/workers/e7240da6-6692-4191-87e1-f1b43e90d950/pids', 'conf': {'supervisor.heartbeat.frequency.secs': 5, 'topology.stats.sample.rate': 0.05, 'topology.fall.back.on.java.serialization': True, 'topology.worker.childopts': '-Xmx128m', 'storm.id': 'wordcount-1-1369714538', 'drpc.worker.threads': 64, 'storm.zookeeper.servers': ['zookeeper'], 'transactional.zookeeper.root': '/transactional', 'supervisor.childopts': '-Xmx256m', 'storm.zookeeper.connection.timeout': 15000, 'zmq.threads': 1, 'topology.executor.send.buffer.size': 1024, 'petrel.host': 'snamenode', 'topology.enable.message.timeouts': True, 'storm.local.mode.zmq': False, 'nimbus.thrift.port': 6627, 'supervisor.worker.timeout.secs': 150, 'nimbus.cleanup.inbox.freq.secs': 600, 'nimbus.topology.validator': 'backtype.storm.nimbus.DefaultTopologyValidator', 'storm.local.dir': '/data/local/storm-local', 'nimbus.supervisor.timeout.secs': 60, 'topology.disruptor.wait.strategy': 'com.lmax.disruptor.BlockingWaitStrategy', 'worker.heartbeat.frequency.secs': 1, 'nimbus.inbox.jar.expiration.secs': 3600, 'supervisor.monitor.frequency.secs': 3, 'drpc.port': 3772, 'topology.trident.batch.emit.interval.millis': 500, 'topology.kryo.factory': 'backtype.storm.serialization.DefaultKryoFactory', 'storm.zookeeper.port': 2181, 'nimbus.childopts': '-Xmx512m', 'topology.workers': 1, 'topology.acker.tasks': None, 'topology.tick.tuple.freq.secs': 150, 'supervisor.worker.start.timeout.secs': 120, 'supervisor.enable': True, 'nimbus.task.launch.secs': 120, 'topology.sleep.spout.wait.strategy.time.ms': 1, 'worker.childopts': '-Xmx128m', 'task.refresh.poll.secs': 10, 'topology.debug': False, 'nimbus.task.timeout.secs': 30, 'topology.message.timeout.secs': 150, 'nimbus.reassign': True, 'petrel.user': 'root', 'drpc.queue.size': 128, 'topology.ackers': 1, 'topology.state.synchronization.timeout.secs': 60, 'topology.executor.receive.buffer.size': 1024, 'nimbus.host': '10.14.21.227', 'ui.port': 28080, 'topology.name': 'wordcount', 'transactional.zookeeper.servers': None, 'drpc.invocations.port': 3773, 'nimbus.file.copy.expiration.secs': 600, 'topology.skip.missing.kryo.registrations': False, 'topology.acker.executors': 1, 'ui.childopts': '-Xmx768m', 'nimbus.monitor.freq.secs': 10, 'topology.max.task.parallelism': None, 'storm.cluster.mode': 'distributed', 'zmq.hwm': 0, 'dev.zookeeper.path': '/data/local/dev-storm-zookeeper', 'drpc.request.timeout.secs': 600, 'zmq.linger.millis': 5000, 'topology.error.throttle.interval.secs': 10, 'topology.worker.shared.thread.pool.size': 4, 'java.library.path': '/usr/local/lib:/opt/local/lib:/usr/lib', 'storm.zookeeper.session.timeout': 20000, 'topology.kryo.register': None, 'storm.zookeeper.retry.times': 5, 'topology.tasks': None, 'storm.zookeeper.root': '/storm', 'topology.spout.wait.strategy': 'backtype.storm.spout.SleepSpoutWaitStrategy', 'topology.max.error.report.per.interval': 5, 'task.heartbeat.frequency.secs': 3, 'supervisor.slots.ports': [6700, 6701, 6702, 6703], 'topology.transfer.buffer.size': 1024, 'topology.max.spout.pending': 1, 'topology.optimize': True, 'topology.kryo.decorators': [], 'storm.zookeeper.retry.interval': 1000, 'transactional.zookeeper.port': None, 'topology.receiver.buffer.size': 8}, 'context': {'task->component': {'1': '__acker', '3': 'split', '2': 'count', '4': 'spout'}}} [2013-05-28 12:16:34,679][storm][ERROR][Errno 32] Broken pipe Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 110, in sendMsgToParent old_stdout.flush() IOError: [Errno 32] Broken pipe [2013-05-28 12:16:34,680][petrel.run][ERROR]E_RUNFAILED_randomsentence_snamenode_7790_StormIPCException [2013-05-28 12:16:34,680][storm][ERROR]Sent failure message ("E_RUNFAILED_randomsentence_snamenode_7790_StormIPCException") to Storm Exception occurred in randomsentence. Worker exiting. Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/run.py", line 100, in main getattr(module, 'run')() File "/data/local/storm-local/supervisor/stormdist/wordcount-1-1369714538/resources/randomsentence.py", line 57, in run RandomSentenceSpout().run() File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 425, in run self.shared_initialize() File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 304, in shared_initialize conf, context = initComponent() File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 271, in initComponent sendpid(setupInfo['pidDir']) File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 143, in sendpid sendMsgToParent({'pid':pid}) File "/usr/local/lib/python2.7/site-packages/petrel-0.8.2.0.1-py2.7.egg/petrel/storm.py", line 116, in sendMsgToParent str(e))) StormIPCException: IOError error [Errno 32] in sendMsgToParent: [Errno 32] Broken pipe (END)
[root@snamenode wordcount-1-1369714538]# cat /root/.pip/pip.log
/tmp/petrel-wordcount-1-1369714538/venv/bin/pip run on Tue May 28 12:16:31 2013 Downloading/unpacking thrift==0.8.0
Getting page https://pypi.python.org/simple/thrift/ Could not fetch URL https://pypi.python.org/simple/thrift/: timed out Will skip URL https://pypi.python.org/simple/thrift/ when looking for download links for thrift==0.8.0 Getting page https://pypi.python.org/simple/ Could not fetch URL https://pypi.python.org/simple/: timed out Will skip URL https://pypi.python.org/simple/ when looking for download links for thrift==0.8.0 Cannot fetch index base URL https://pypi.python.org/simple/
URLs to search for versions for thrift==0.8.0:
https://pypi.python.org/simple/thrift/0.8.0 https://pypi.python.org/simple/thrift/ Getting page https://pypi.python.org/simple/thrift/0.8.0 Getting page https://pypi.python.org/simple/thrift/ Could not fetch URL https://pypi.python.org/simple/thrift/0.8.0: timed out Will skip URL https://pypi.python.org/simple/thrift/0.8.0 when looking for download links for thrift==0.8.0 Could not fetch URL https://pypi.python.org/simple/thrift/: timed out Will skip URL https://pypi.python.org/simple/thrift/ when looking for download links for thrift==0.8.0 Could not find any downloads that satisfy the requirement thrift==0.8.0 No distributions at all found for thrift==0.8.0
Exception information: Traceback (most recent call last): File "/tmp/petrel-wordcount-1-1369714538/venv/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main status = self.run(options, args) File "/tmp/petrel-wordcount-1-1369714538/venv/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 266, in run requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) File "/tmp/petrel-wordcount-1-1369714538/venv/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 1026, in prepare_files url = finder.find_requirement(req_to_install, upgrade=self.upgrade) File "/tmp/petrel-wordcount-1-1369714538/venv/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/index.py", line 171, in find_requirement raise DistributionNotFound('No distributions at all found for %s' % req) DistributionNotFound: No distributions at all found for thrift==0.8.0 [root@snamenode wordcount-1-1369714538]# ll
[root@snamenode logs]# tail -f supervisor.log nt-1-1369714538/stormjar.jar backtype.storm.daemon.worker wordcount-1-1369714538 6d6bbfa8-4687-48f2-86a5-53a7ad828ace 6703 0f06a4f7-d487-4403-95c0-1b2d9e155c54 2013-05-28 12:54:27 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:28 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:28 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:29 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:29 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:30 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:30 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started 2013-05-28 12:54:31 supervisor [INFO] 0f06a4f7-d487-4403-95c0-1b2d9e155c54 still hasn't started
form storm ui error
java.lang.RuntimeException: Pipe to subprocess seems to be broken! No output read. Shell Process Exception:
at backtype.storm.utils.ShellProcess.readString(ShellProcess.java:100) at backtype.storm.utils.ShellProcess.readMessage(ShellProcess.java:62) at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:43) at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:79) at backtype.storm.daemon.executor$fn4050$fn4059.invoke(executor.clj:610) at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:662)