'Estimator job failed' while using EstimatorQNN to train NN using TorchConnector

[Next-di-mension]

I am using EstimatorQNN to train a NN using TorchConnector.

This is how I define the QNN:

input_dim = int(np.log2(feature_dim))
feature_map = RawFeatureVector(feature_dimension=feature_dim) # amplitude encoding 
ansatz = RealAmplitudes(num_qubits=input_dim, entanglement='full', reps=2)
qc = QuantumCircuit(input_dim)
qc.compose(feature_map, inplace=True)
qc.compose(ansatz, inplace=True)

qnn = EstimatorQNN(
    circuit=qc, input_params=feature_map.parameters, weight_params=ansatz.parameters
)
initial_weights = 0.1 * (2 * algorithm_globals.random.random(qnn.num_weights) - 1)
model_connected = TorchConnector(qnn, initial_weights=initial_weights)

while training it using PyTorch, I am using the NLLLoss function. When I give a loss.backward() command, it throws an error 'Estimator job failed' Can somebody help me with this?

[Next-di-mension]

Here is the working example and the error message: example:

import numpy as np
import matplotlib.pyplot as plt

import torch 
import torch.nn as nn
from torch import Tensor
from torch.nn import Linear, CrossEntropyLoss, MSELoss
from torch.optim import LBFGS
from torch.nn import ReLU, Tanh, Sigmoid, ELU, LeakyReLU, PReLU

from qiskit import QuantumCircuit
from qiskit.utils import algorithm_globals
from qiskit.circuit import Parameter
from qiskit.circuit.library import RealAmplitudes, ZZFeatureMap
from qiskit_machine_learning.neural_networks import SamplerQNN, EstimatorQNN
from qiskit_machine_learning.connectors import TorchConnector
from qiskit_machine_learning.circuit.library import RawFeatureVector

# Set up a circuit
feature_dim = 256
input_dim = int(np.log2(feature_dim))
feature_map = RawFeatureVector(feature_dimension=feature_dim) # amplitude encoding 
ansatz = RealAmplitudes(num_qubits=input_dim, entanglement='full', reps=2)
qc = QuantumCircuit(input_dim)
qc.compose(feature_map, inplace=True)
qc.compose(ansatz, inplace=True)

qnn = EstimatorQNN(
    circuit=qc, input_params=feature_map.parameters, weight_params=ansatz.parameters
)
initial_weights = 0.1 * (2 * algorithm_globals.random.random(qnn.num_weights) - 1)
model1 = TorchConnector(qnn, initial_weights=initial_weights) # connects the qnn with pytorch 

input_size = 256
num_classes = 256
x = torch.randn(256)
y = torch.randn(256)
class ClassicalNN(nn.Module):
    def __init__(self, input_size, num_classes):
        super(ClassicalNN, self).__init__()

        self.fc = nn.Linear(input_size, num_classes) 

    def forward(self, x):
        x = self.fc(x)
        return x

# Set up the hybrid model combining classical and quantum components
class HybridModel(nn.Module):
    def __init__(self, input_size, num_classes):
        super(HybridModel, self).__init__()
        self.classical = ClassicalNN(input_size, num_classes)
        self.quantum = model1

    def forward(self, x):
        x = self.classical(x)
        x = self.quantum(x)
        return x

model2 = HybridModel(input_size = 256, num_classes = 256)   

optimizer = LBFGS(model2.parameters())
f_loss = MSELoss(reduction="sum")

model2.train() 

def closure():
    optimizer.zero_grad()  # Initialize/clear gradients
    loss = f_loss(model2(x), y)  # Evaluate loss function
    loss.backward()  # Backward pass
    print(loss.item())  # Print loss
    return loss

# Run optimizer step4
optimizer.step(closure)

error:

---------------------------------------------------------------------------
QiskitError                               Traceback (most recent call last)
File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit_machine_learning\neural_networks\estimator_qnn.py:245](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit_machine_learning/neural_networks/estimator_qnn.py:245), in EstimatorQNN._backward(self, input_data, weights)
    244 try:
--> 245     results = job.result()
    246 except Exception as exc:

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\primitives\primitive_job.py:55](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/primitives/primitive_job.py:55), in PrimitiveJob.result(self)
     54 self._check_submitted()
---> 55 return self._future.result()

File [C:\Program](file:///C:/Program) Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\_base.py:437, in Future.result(self, timeout)
    436 elif self._state == FINISHED:
--> 437     return self.__get_result()
    439 self._condition.wait(timeout)

File [C:\Program](file:///C:/Program) Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\_base.py:389, in Future.__get_result(self)
    388 try:
--> 389     raise self._exception
    390 finally:
    391     # Break a reference cycle with the exception in self._exception

File [C:\Program](file:///C:/Program) Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\thread.py:57, in _WorkItem.run(self)
     56 try:
---> 57     result = self.fn(*self.args, **self.kwargs)
     58 except BaseException as exc:

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\algorithms\gradients\param_shift_estimator_gradient.py:67](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/algorithms/gradients/param_shift_estimator_gradient.py:67), in ParamShiftEstimatorGradient._run(self, circuits, observables, parameter_values, parameters, **options)
     66 """Compute the gradients of the expectation values by the parameter shift rule."""
---> 67 g_circuits, g_parameter_values, g_parameters = self._preprocess(
     68     circuits, parameter_values, parameters, self.SUPPORTED_GATES
     69 )
     70 results = self._run_unique(
     71     g_circuits, observables, g_parameter_values, g_parameters, **options
     72 )

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\algorithms\gradients\base_estimator_gradient.py:195](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/algorithms/gradients/base_estimator_gradient.py:195), in BaseEstimatorGradient._preprocess(self, circuits, parameter_values, parameters, supported_gates)
    194 if circuit_key not in self._gradient_circuit_cache:
--> 195     unrolled = translator(circuit)
    196     self._gradient_circuit_cache[circuit_key] = _assign_unique_parameters(unrolled)

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\transpiler\basepasses.py:121](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/transpiler/basepasses.py:121), in BasePass.__call__(self, circuit, property_set)
    119     self.property_set = property_set_
--> 121 result = self.run(circuit_to_dag(circuit))
    123 result_circuit = circuit

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\transpiler\passes\basis\translate_parameterized.py:136](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/transpiler/passes/basis/translate_parameterized.py:136), in TranslateParameterizedGates.run(self, dag)
    133 if _is_parameterized(node.op) and not _is_supported(
    134     node, self._supported_gates, self._target
    135 ):
--> 136     definition = node.op.definition
    138     if definition is not None:
    139         # recurse to unroll further parameterized blocks

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit\circuit\instruction.py:239](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit/circuit/instruction.py:239), in Instruction.definition(self)
    238 if self._definition is None:
--> 239     self._define()
    240 return self._definition

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit_machine_learning\circuit\library\raw_feature_vector.py:170](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit_machine_learning/circuit/library/raw_feature_vector.py:170), in ParameterizedInitialize._define(self)
    169     else:
--> 170         raise QiskitError("Cannot define a ParameterizedInitialize with unbound parameters")
    172 # normalize

QiskitError: 'Cannot define a ParameterizedInitialize with unbound parameters'

The above exception was the direct cause of the following exception:

QiskitMachineLearningError                Traceback (most recent call last)
Cell In[23], line 78
     75     return loss
     77 # Run optimizer step4
---> 78 optimizer.step(closure)

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\optim\optimizer.py:280](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/optim/optimizer.py:280), in Optimizer.profile_hook_step..wrapper(*args, **kwargs)
    276         else:
    277             raise RuntimeError(f"{func} must return None or a tuple of (new_args, new_kwargs),"
    278                                f"but got {result}.")
--> 280 out = func(*args, **kwargs)
    281 self._optimizer_step_code()
    283 # call optimizer step post hooks

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\utils\_contextlib.py:115](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/utils/_contextlib.py:115), in context_decorator..decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\optim\lbfgs.py:312](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/optim/lbfgs.py:312), in LBFGS.step(self, closure)
    309 state.setdefault('n_iter', 0)
    311 # evaluate initial f(x) and df/dx
--> 312 orig_loss = closure()
    313 loss = float(orig_loss)
    314 current_evals = 1

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\utils\_contextlib.py:115](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/utils/_contextlib.py:115), in context_decorator..decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

Cell In[23], line 73, in closure()
     71 optimizer.zero_grad()  # Initialize/clear gradients
     72 loss = f_loss(model2(x), y)  # Evaluate loss function
---> 73 loss.backward()  # Backward pass
     74 print(loss.item())  # Print loss
     75 return loss

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\_tensor.py:487](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/_tensor.py:487), in Tensor.backward(self, gradient, retain_graph, create_graph, inputs)
    477 if has_torch_function_unary(self):
    478     return handle_torch_function(
    479         Tensor.backward,
    480         (self,),
   (...)
    485         inputs=inputs,
    486     )
--> 487 torch.autograd.backward(
    488     self, gradient, retain_graph, create_graph, inputs=inputs
    489 )

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\autograd\__init__.py:200](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/autograd/__init__.py:200), in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    195     retain_graph = create_graph
    197 # The reason we repeat same the comment below is that
    198 # some Python versions print out the first line of a multi-line function
    199 # calls in the traceback and some print out the last line
--> 200 Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    201     tensors, grad_tensors_, retain_graph, create_graph, inputs,
    202     allow_unreachable=True, accumulate_grad=True)

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\autograd\function.py:274](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/torch/autograd/function.py:274), in BackwardCFunction.apply(self, *args)
    270     raise RuntimeError("Implementing both 'backward' and 'vjp' for a custom "
    271                        "Function is not allowed. You should only implement one "
    272                        "of them.")
    273 user_fn = vjp_fn if vjp_fn is not Function.vjp else backward_fn
--> 274 return user_fn(self, *args)

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit_machine_learning\connectors\torch_connector.py:171](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit_machine_learning/connectors/torch_connector.py:171), in TorchConnector._TorchNNFunction.backward(ctx, grad_output)
    168     grad_output = grad_output.view(1, -1)
    170 # evaluate QNN gradient
--> 171 input_grad, weights_grad = neural_network.backward(
    172     input_data.detach().cpu().numpy(), weights.detach().cpu().numpy()
    173 )
    174 if input_grad is not None:
    175     if ctx.sparse:

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit_machine_learning\neural_networks\neural_network.py:254](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit_machine_learning/neural_networks/neural_network.py:254), in NeuralNetwork.backward(self, input_data, weights)
    252 input_, shape = self._validate_input(input_data)
    253 weights_ = self._validate_weights(weights)
--> 254 input_grad, weight_grad = self._backward(input_, weights_)
    256 input_grad_reshaped, weight_grad_reshaped = self._validate_backward_output(
    257     input_grad, weight_grad, shape
    258 )
    260 return input_grad_reshaped, weight_grad_reshaped

File [~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\qiskit_machine_learning\neural_networks\estimator_qnn.py:247](https://file+.vscode-resource.vscode-cdn.net/e%3A/GNN-VQC/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0/LocalCache/local-packages/Python38/site-packages/qiskit_machine_learning/neural_networks/estimator_qnn.py:247), in EstimatorQNN._backward(self, input_data, weights)
    245             results = job.result()
    246         except Exception as exc:
--> 247             raise QiskitMachineLearningError("Estimator job failed.") from exc
    249         input_grad, weights_grad = self._backward_postprocess(num_samples, results)
    251 return input_grad, weights_grad

QiskitMachineLearningError: 'Estimator job failed.'

[poojithumeshrao]

I am facing the same issue. Can you please share if you found any workaround for it?

[Next-di-mension]

No, I am still facing this issue.

[tasnimahmed11]

Hey! I faced a similar issue and thought it had something to do with the Torchconnector and my inputs not being formatted correctly. However, in my circuit, the inputs and weights were not bound properly to the circuit. Have a look into the number of parameters both your feature map and ansatz has and then map that back to the number of inputs and weights you are sending in.

[Next-di-mension]

Hey, thank you for your reply! I checked the number of parameters (both feature map and ansatz params) and they match exactly with the number of input features and weights respectively. Is there something that I am missing? What did you mean when you said the inputs and weights were not bound properly to the circuit ?

[kshitijdave]

Hey, thank you for your reply! I checked the number of parameters (both feature map and ansatz params) and they match exactly with the number of input features and weights respectively. Is there something that I am missing? What did you mean when you said the inputs and weights were not bound properly to the circuit ?

Hey did you find any solution??

[kshitijdave]

I am facing this issue for past 3 days and it happens in sampler and estimator only one when i tried to integrate with noise model. In EstimatorQNN with esttimator parameter and in sampler with sampler parameters.

[Next-di-mension]

i was facing the issue when i tried to integrate it with torchconnector

[floriankittelmann]

I am facing the same problem using SamplerQNN and tried to debug.

The problem happens in method _preprocess of BaseSamplerGradient, which gets called for gradient calculation during the backwards call. I guess there is a according method in BaseEstimatorGradient, but I didn't check. For using TranslateParameterizedGates on a quantum circuit, firstly the parameter values for RawFeatureVector needs to be assigned, otherwise the QiskitError "Cannot define a ParameterizedInitialize with unbound parameters" is raised. But the parameters are not assign in BaseSamplerGradient before calling TranslateParameterizedGates on the circuit. If there is just angle encoding in the circuit, TranslateParameterizedGates works fine without assigning the parameter values first.

Can this be fixed in future releases? In my opinion, when using EstimatorQNN or SamplerQNN with the option input_gradients=False, it should be able to calculate gradients with respect to the weights even if RawFeatureVector is used.

Please see my example:

from qiskit.circuit.library import ZZFeatureMap
from qiskit_machine_learning.circuit.library import RawFeatureVector
from qiskit import QuantumCircuit
from qiskit.transpiler.passes import TranslateParameterizedGates
import numpy as np

SUPPORTED_GATES = [
        "x",
        "y",
        "z",
        "h",
        "rx",
        "ry",
        "rz",
        "p",
        "cx",
        "cy",
        "cz",
        "ryy",
        "rxx",
        "rzz",
        "rzx",
    ]
translator = TranslateParameterizedGates(SUPPORTED_GATES)

# works fine
qc = QuantumCircuit(8)
qc = qc.compose(ZZFeatureMap(8))
qc.measure_all()
translator(qc)

# raises QiskitError('Cannot define a ParameterizedInitialize with unbound parameters')
qc = QuantumCircuit(3)
qc = qc.compose(RawFeatureVector(8))
qc.measure_all()
translator(qc)

# now RawFeatureVector works
qc = QuantumCircuit(3)
qc = qc.compose(RawFeatureVector(8))
qc.measure_all()
features = np.random.rand(8)
qc = qc.assign_parameters(features)
translator(qc)

[FrancescaSchiav]

Hi @Next-di-mension, thanks for opening this issue, it seems a number of people have undergone it as well.

I can reproduce your error by switching out the: from qiskit.utils import algorithm_globals to from qiskit_algorithms.utils import algorithm_globals or by using either numpy or random random number generators, as the algorithm_globals from qiskit.utils has been deprecated.

Whilst tracing back through your error message I can see that the error is not really with the Estimator (nor would it be with Sampler) or Torch Connector, but with the binding of parameters, which someone in the comments also mentioned.

If you look at the Raw Feature Vector class (https://qiskit-community.github.io/qiskit-machine-learning/stubs/qiskit_machine_learning.circuit.library.RawFeatureVector.html) you will see it specifically states that: "The circuit contains a placeholder instruction that can only be synthesized/defined when all parameters are bound". And that is where QiskitError("Cannot define a ParameterizedInitialize with unbound parameters") is raised, but the code is not interrupted until later, but it is the direct cause of the error "Estimator job Failed". You can use the assign_parameters function to bind parameters. You can check out the source code as well (https://qiskit-community.github.io/qiskit-machine-learning/_modules/qiskit_machine_learning/circuit/library/raw_feature_vector.html#RawFeatureVector).

In the documentation it also states that the circuit resulting from Raw Feature Vector cannot be used with gradient based optimizers as gradients can't be computed. So at the moment we are not going to change EstimatorQNN or SamplerQNN as suggested "with the option input_gradients=False, it should be able to calculate gradients with respect to the weights even if RawFeatureVector is used".

If you have other queries or comments let us know, and if you run into other errors related to this you are welcome to open a new issue as well.