vqe_runtime gives serialization error with optimizer COBYLA [BUG]

[qcabepsilon]

Expected behavior

When using this tutorial for using pennylane-qiskit vqe_runtime_runner.py to run vqe_runtime_program.py I would expect all optimizers of SciPy can be used.

Actual behavior

When specifying 'COBYLA' as optimizer the job fails with the message TypeError: Object of type bool_ is not JSON serializable.

Additional information

No response

Source code

See link to tutorial.

Tracebacks

RuntimeJobFailureError                    Traceback (most recent call last)
/home/.../send.py in <module>
     25 hamiltonian = qml.Hamiltonian(coeffs, obs)
     27 job = vqe_rt_runner.vqe_runner(
     28     program_id=program_id,
     29     backend="ibmq_qasm_simulator",
   (...)
     36     kwargs={"hub": "ibm-q", "group": "open", "project": "main"},
     37 )
---> 39 print(job.result())

File ~/..../vqe_runtime_runner.py:106, in RuntimeJobWrapper.result(self)
     98 def result(self):
     99     """Get the result of the job as a SciPy OptimizerResult object.
    100 
    101     This method blocks until the job is done, cancelled, or raises an error.
   (...)
    104         OptimizerResult: An optimizer result object.
    105     """
--> 106     return self._job.result(decoder=self._decoder)

File ~/Project/venv/lib/python3.8/site-packages/qiskit/providers/ibmq/runtime/runtime_job.py:151, in RuntimeJob.result(self, timeout, wait, decoder)
    149 self.wait_for_final_state(timeout=timeout, wait=wait)
    150 if self._status == JobStatus.ERROR:
--> 151     raise RuntimeJobFailureError(f"Unable to retrieve job result. "
    152                                  f"{self.error_message()}")
    153 result_raw = self._api_client.job_results(job_id=self.job_id())
    154 self._results = _decoder.decode(result_raw)

Log:
RuntimeJobFailureError: 'Unable to retrieve job result. Job ccu4c7tobjier7iv9bcg has failed. ... starter.execute()\n2022-10-04T14:41:42.xxx   File "/provider/programruntime/program_starter_wrapper.py", line 99, in execute\n2022-10-04T14:41:42.xxx     raise ex\n2022-10-04T14:41:42.xxx   File "/provider/programruntime/program_starter_wrapper.py", line 91, in execute\n2022-10-04T14:41:42.xxx     self.messenger.publish(final_result, final=True)\n2022-10-04T14:41:42.xxx   File "/provider/programruntime/runtime_user_messenger.py", line 59, in publish\n2022-10-04T14:41:42.xxx     message = json.dumps(message, cls=encoder)\n2022-10-04T14:41:42.xxx   File "/usr/lib64/python3.8/json/__init__.py", line 234, in dumps\n2022-10-04T14:41:42.xxx     return cls(\n2022-10-04T14:41:42.xxx   File "/usr/lib64/python3.8/json/encoder.py", line 199, in encode\n2022-10-04T14:41:42.xxx     chunks = self.iterencode(o, _one_shot=True)\n2022-10-04T14:41:42.xxx   File "/usr/lib64/python3.8/json/encoder.py", line 257, in iterencode\n2022-10-04T14:41:42.xxx     return _iterencode(o, 0)\n2022-10-04T14:41:42.xxx   File "/opt/app-root/lib64/python3.8/site-packages/qiskit_ibm_runtime/utils/json.py", line 261, in default\n2022-10-04T14:41:42.xxx     return super().default(obj)\n2022-10-04T14:41:42.xxx   File "/usr/lib64/python3.8/json/encoder.py", line 179, in default\n2022-10-04T14:41:42.xxx     raise TypeError(f\'Object of type {o.__class__.__name__} \'\n2022-10-04T14:41:42.xxx TypeError: Object of type bool_ is not JSON serializable\n'

System information

Name: PennyLane
Version: 0.26.0
Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
Home-page: https://github.com/XanaduAI/pennylane
Author: None
Author-email: None
License: Apache License 2.0
Location: /home/cai/ProjectLocal/venvLocal3/lib/python3.8/site-packages
Requires: retworkx, pennylane-lightning, toml, cachetools, appdirs, semantic-version, networkx, autograd, scipy, numpy, autoray
Required-by: PennyLane-qiskit, PennyLane-Qchem, PennyLane-Lightning

Platform info:           Linux-5.4.0-121-generic-x86_64-with-glibc2.29
Python version:          3.8.10
Numpy version:           1.22.2
Scipy version:           1.8.0
Installed devices:
- default.gaussian (PennyLane-0.26.0)
- default.mixed (PennyLane-0.26.0)
- default.qubit (PennyLane-0.26.0)
- default.qubit.autograd (PennyLane-0.26.0)
- default.qubit.jax (PennyLane-0.26.0)
- default.qubit.tf (PennyLane-0.26.0)
- default.qubit.torch (PennyLane-0.26.0)
- default.qutrit (PennyLane-0.26.0)
- qiskit.aer (PennyLane-qiskit-0.21.0)
- qiskit.basicaer (PennyLane-qiskit-0.21.0)
- qiskit.ibmq (PennyLane-qiskit-0.21.0)
- qiskit.ibmq.circuit_runner (PennyLane-qiskit-0.21.0)
- qiskit.ibmq.sampler (PennyLane-qiskit-0.21.0)
- lightning.qubit (PennyLane-Lightning-0.26.0)

vqe_runtime_runner.py (with line 468 changed to coeff, observables= hamiltonian.terms) and vqe_runtime_program.py from https://docs.pennylane.ai/projects/qiskit/en/latest/_modules/index.html, 4th Okt 2022

Existing GitHub issues

• [X] I have searched existing GitHub issues to make sure the issue does not already exist.

[josh146]

Thanks for reporting this @qcabepsilon! I have a feeling this might be related to this question as well: https://quantumcomputing.stackexchange.com/questions/28399/difference-between-qiskit-spsa-optimize-and-minimize-in-runtime-program

[rmoyard]

Hi @qcabepsilon, thanks for pointing this issue. I don't know why the success attribute of Cobyla OptimizerResult is a numpy bool which is not serializable. I created a PR that convert that to a conventional bool. It should fix the issue. https://github.com/PennyLaneAI/pennylane-qiskit/pull/231

[qcabepsilon]

Thanks for reporting this @qcabepsilon! I have a feeling this might be related to this question as well: https://quantumcomputing.stackexchange.com/questions/28399/difference-between-qiskit-spsa-optimize-and-minimize-in-runtime-program

It might be. Should I create a new issue for that, so this one can be closed?

[rmoyard]

I can answer your question here you don't need to open another one. The SPSA class used is not from Scipy but from Qiskit: https://qiskit.org/documentation/stubs/qiskit.algorithms.optimizers.SPSA.html#qiskit.algorithms.optimizers.SPSA . It does not contain a minimize method and does not store the result in the scipy OptimizerResult. Therefore we have to create this object with the result of Qiskit SPSA method optimize.

[qcabepsilon]

The documentation at the link you shared states that the SPSA of qiskit has the method minimize which returns its results as OptimizerResult object. What are the differences between this class and the SciPy class of the same name? The optimize method uses minimize itself, but returns the result as single values instead of the object. I do not understand why one of them lead to problems in the vqe_runtime_program.py and vqe_runtime_runner.py

[rmoyard]

I don't know the exact details on why but Qiskit decided to implement its own SPSA optimizer, it is probably because they wanted some flexibility in the argument. For the vqe_runtime we decided to have only one type of result which is a scipy OptimizerResult that contains not only the results but the parameters, success, and other attributes. Therefore for SPSA and QNSPSA (Qiskit implemented) we have to take create the scipy OptimizerResult.

[qcabepsilon]

I assumed that the qiskit OptimizerResult class and the one from SciPy are the same. Thank you for resolving this misconception. One more question: Why does the conversion to the SciPy OptimizerResult for NFT not work as the one for the SPSA optimizer? (see stackexchange question)

[rmoyard]

The only qiskit optimizers that we currently support are SPSA and QNSPSA but you should able to adapt it like we did for the those two. Potentially some arguments are different and it does not work because of that, or maybe some output are different. It is also possible that NFT is not adapted for your problem. You can share your code and error messages as well here.

[qcabepsilon]

I figured the problem out. Thank you very much for your help.