search

radical-collaboration / extasy-grlsd

Repository to hold the input data and scripts for the ExTASY gromacs-lsdmap work
1 stars 1 forks source link

client doesn't stop #100

Closed euhruska closed 5 years ago

euhruska commented 5 years ago

After the pbs job finishes the client still continues doing heartbeat and doesn't automatically finish. I still can kill it manually but it should do it automatically. What are the possible reasons?

euhruska commented 5 years ago

Now it stopped after throwing this exception:

2018-10-17 20:17:36,042: radical.entk.task_manager.0000: MainProcess                     : heartbeat      : ERROR   : Heartbeat failed with error: (404, "NOT_FOUND - no queue 're.session.leon
ardo.rice.edu.eh22.017821.0002-hb-response' in vhost '/'")
Traceback (most recent call last):
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/radical/entk/execman/base/task_manager.py", line 152, in _heartbeat
    method_frame, props, body = mq_channel.basic_get(queue=self._hb_response_q)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 2077, in basic_get
    self._basic_getempty_result.is_ready)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 1292, in _flush_output
    *waiters)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 458, in _flush_output
    self._impl.ioloop.poll()
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/select_connection.py", line 495, in poll
    self._poller.poll()
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/select_connection.py", line 1114, in poll
    self._dispatch_fd_events(fd_event_map)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/select_connection.py", line 831, in _dispatch_fd_events
    handler(fileno, events)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/base_connection.py", line 410, in _handle_events
    self._handle_read()
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/base_connection.py", line 464, in _handle_read
    self._on_data_available(data)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/connection.py", line 2021, in _on_data_available
    self._process_frame(frame_value)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/connection.py", line 2142, in _process_frame
    if self._process_callbacks(frame_value):
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/connection.py", line 2123, in _process_callbacks
    frame_value)  # Args
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/callback.py", line 60, in wrapper
    return function(*tuple(args), **kwargs)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/callback.py", line 92, in wrapper
    return function(*args, **kwargs)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/callback.py", line 236, in process
    callback(*args, **keywords)
  File "/scratch1/eh22/conda/envs/extasy11/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 1358, in _on_channel_closed
    method.reply_text)
ChannelClosed: (404, "NOT_FOUND - no queue 're.session.leonardo.rice.edu.eh22.017821.0002-hb-response' in vhost '/'")