search

galaxyproject / pulsar

Distributed job execution application built for Galaxy
https://pulsar.readthedocs.io
Apache License 2.0
37 stars 50 forks source link

assign_ids: uuid produces an IOError: [Errno 2] No such file or directory #194

Closed gmauro closed 3 years ago

gmauro commented 5 years ago

I have set assign_ids: uuid in the app.yml file and the result has been an IOError: [Errno 2] No such file or directory:

Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: 2019-06-17 14:36:41,898 ERROR [pulsar.messaging.bind_amqp][consume-setup-pyamqp://galaxy_freiburg:********@proxy.internal.galaxyproject.eu:5671//pulsar/galaxy_freiburg?ssl=1] Failed to setup job 161 obtained via message queue.
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: Traceback (most recent call last):
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/messaging/bind_amqp.py", line 117, in __process_setup_message
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: manager_endpoint_util.submit_job(manager, body)
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/manager_endpoint_util.py", line 102, in submit_job
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: manager.preprocess_and_launch(job_id, launch_config)
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/managers/stateful.py", line 85, in preprocess_and_launch
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: self._persist_launch_config(job_id, launch_config)
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/managers/stateful.py", line 76, in _persist_launch_config
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: job_directory.store_metadata("launch_config", launch_config)
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/managers/base/__init__.py", line 330, in store_metadata
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: self.write_file(metadata_name, json.dumps(metadata_value))
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: File "/opt/pulsar/venv/lib/python2.7/site-packages/pulsar/managers/base/__init__.py", line 256, in write_file
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: job_file = open(path, 'wb')
Jun 17 14:36:41 vgcn-central-manager.usegalaxy.eu.novalocal pulsar[333945]: IOError: [Errno 2] No such file or directory: u'/data/share/staging/161/launch_config'

/data/share/staging/161 should be /data/share/staging/e555002620574d77ad3685ffb12645aa

jmchilton commented 5 years ago

Are you using __PULSAR_JOBS_DIRECTORY__ - do you have to be using that option?

gmauro commented 5 years ago

No, I'm not using that option.

hexylena commented 3 years ago

Probably solved with above comment. @gmauro reopen if not :)