diego-plan9
commented
5 years ago I have only encountered this a few times, but it seems sort of random as to when it occurs.
Unfortunately it seems to be indeed due to network issues - could it be that you are running several jobs against IBMQ (ie. against an online device or simulator) concurrently, or in a short time span? There are some network limits and availability constraints that might be in play in your current scenario.
Informations
What is the current behavior?
When using the backend ibmq_qasm_simulator for a qaoa script, two different errors are encountered. The first one is just ignored and the script continues running, the second causes it to stop.
Steps to reproduce the problem
I have only encountered this a few times, but it seems sort of random as to when it occurs. I am testing some QAOA scripts using both IBM Quantum Experience and Rigetti Simulator, all of the current code is on this directory - https://github.com/epelofske/quantum_optimization/blob/master. To reproduce, running combined_qaoa_test.py will likely get the error eventually. I have only tested using the backend ibmq_qasm_simulator, but it might occur when using other backends.
What is the expected behavior?
The following two errors are encountered:
Got a 502 code response to /api/Jobs/5c23aaa40fc4f2005836cb6f/status: 502 Bad Gateway: Registered endpoint failed to handle the request.
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request self._validate_conn(conn) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 852, in _validate_conn conn.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 284, in connect conn = self._new_conn() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7ff51940e898>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 388, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='quantumexperience.ng.bluemix.net', port=443): Max retries exceeded with url: /api/Jobs/5c23cc6f5259d20051b060d3/status?access_token=CENSORED (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff51940e898>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/root/all/qiskit/providers/ibmq/ibmqjob.py", line 267, in status api_job = self._api.get_status_job(self._job_id) File "/root/all/qiskit/providers/ibmq/api/ibmqconnector.py", line 674, in get_status_job status = self.req.get(url) File "/root/all/qiskit/providers/ibmq/api/ibmqconnector.py", line 357, in get self.extra_args) File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 75, in get return request('get', url, params=params, kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='quantumexperience.ng.bluemix.net', port=443): Max retries exceeded with url: /api/Jobs/5c23cc6f5259d20051b060d3/status?access_token= CENSORED (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff51940e898>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "combined_qaoa_test.py", line 80, in
print(combined_test2('agave_ibmqx_no_noise.txt'))
File "combined_qaoa_test.py", line 72, in combined_qaoa_test
x = solve_ising_qubo(G, max_clique_qubo_matrix_ibmqx, optimizer, a)
File "/root/all/ibmqx_ising_qubo_qaoa.py", line 65, in solve_ising_qubo
result = qaoa.run(quantum_instance)
File "/root/all/qiskit_aqua/algorithms/quantum_algorithm.py", line 103, in run
return self._run()
File "/root/all/qiskit_aqua/algorithms/adaptive/vqe/vqe.py", line 264, in _run
self._solve()
File "/root/all/qiskit_aqua/algorithms/adaptive/vqe/vqe.py", line 208, in _solve
opt_params, opt_val = self.find_minimum_eigenvalue()
File "/root/all/qiskit_aqua/algorithms/adaptive/vqe/vqe.py", line 348, in find_minimum_eigenvalue
variable_bounds=bounds, initial_point=initial_point)
File "/root/all/qiskit_aqua/components/optimizers/slsqp.py", line 87, in optimize
options=self._options)
File "/usr/lib/python3/dist-packages/scipy/optimize/_minimize.py", line 611, in minimize
constraints, callback=callback, *options)
File "/usr/lib/python3/dist-packages/scipy/optimize/slsqp.py", line 403, in _minimize_slsqp
g = append(fprime(x), 0.0)
File "/usr/lib/python3/dist-packages/scipy/optimize/optimize.py", line 293, in function_wrapper
return function((wrapper_args + args))
File "/usr/lib/python3/dist-packages/scipy/optimize/slsqp.py", line 63, in approx_jacobian
jac[i] = (func(((x0+dx,)+args)) - f0)/epsilon
File "/usr/lib/python3/dist-packages/scipy/optimize/optimize.py", line 293, in function_wrapper
return function((wrapper_args + args))
File "/root/all/qiskit_aqua/algorithms/adaptive/vqe/vqe.py", line 291, in _energy_evaluation
result = self._quantum_instance.execute(to_be_simulated_circuits)
File "/root/all/qiskit_aqua/quantum_instance.py", line 156, in execute
has_shared_circuits=self._shared_circuits)
File "/root/all/qiskit_aqua/utils/run_circuits.py", line 268, in compile_and_run_circuits
results.append(job.result(**qjob_config))
File "/root/all/qiskit/providers/ibmq/ibmqjob.py", line 203, in result
job_response = self._wait_for_result(timeout=timeout, wait=wait)
File "/root/all/qiskit/providers/ibmq/ibmqjob.py", line 210, in _wait_for_result
job_response = self._wait_for_job(timeout=timeout, wait=wait)
File "/root/all/qiskit/providers/ibmq/ibmqjob.py", line 389, in _wait_for_job
while self.status() not in JOB_FINAL_STATES:
File "/root/all/qiskit/providers/ibmq/ibmqjob.py", line 273, in status
raise JobError(str(err))
qiskit.providers.exceptions.JobError: "HTTPSConnectionPool(host='quantumexperience.ng.bluemix.net', port=443): Max retries exceeded with url: /api/Jobs/5c23cc6f5259d20051b060d3/status?access_token=CENSORED (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff51940e898>: Failed to establish a new connection: [Errno 110] Connection timed out',))"
Suggested solutions
I am not familiar enough with network error handling to know what solution could work. In the meantime, I am trying just using local simulators, and also eventually an IBMQ NISQ device to see if the same error occurs.