long waits on keyboard interrupt

boothby commented 6 years ago

Using dwave.system.samplers.DWaveSampler, I occasionally want to stop the process I'm running with a KeyboardInterrupt. This often takes upwards of 10 seconds... which seems a little excessive. Here's the traceback, when the interrupt finally gets caught.

    response.change_vartype(Vartype.SPIN, data_vector_offsets={'energy': offset})
  File "/usr/local/lib/python2.7/dist-packages/dimod/decorators.py", line 173, in new_f
    return f(*final_args, **final_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/dimod/response.py", line 803, in change_vartype
    self.data_vectors[key] += data_vector_offsets[key]
  File "/usr/local/lib/python2.7/dist-packages/dimod/response.py", line 211, in data_vectors
    self._resolve_futures(**self._futures)
  File "/usr/local/lib/python2.7/dist-packages/dimod/response.py", line 588, in _resolve_futures
    for future in as_completed(futures):
  File "/usr/local/lib/python2.7/dist-packages/dwave/cloud/computation.py", line 311, in as_completed
    done, not_done = Future.wait_multiple(not_done, min_done=1, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/dwave/cloud/computation.py", line 257, in wait_multiple
    event.wait(wait_time)
  File "/usr/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/lib/python2.7/threading.py", line 340, in wait
    waiter.acquire()
KeyboardInterrupt

randomir commented 6 years ago

Known undesirable behaviour, but thanks for reporting. The fix is planned (depends on another refactor).

The reason for this is we currently wait for future(s) to resolve (task queue empties) before joining. Instead we should catch Ctrl+C and kill all child workers.

davage commented 5 years ago

I have seen similar behaviour when using the dwave ping command on Linux. When my account is out of credit, the client submits a problem and waits for a response, but the problem stays in the pending state until it is cancelled or more credit is available. This results in the the process blocking the terminal session. The ctrl+c command won't stop the process from running. As a workaround, I was able to find the process using ps aux | grep "dwave ping" and stop it using the kill <pid> command.

randomir commented 5 years ago

@davage, nice catch, we should look into handling out-of-quota scenarios.

dwavesystems / dwave-cloud-client

long waits on keyboard interrupt #225