Key error in Accuracy function

[mohapatra-amrit]

Hi guys. I have executed these codes properly, print('Final test accuracy: {}'.format(accuracy(test_pair_circuits, test_labels))) is not working.

def accuracy(circs, labels): predicted = model(circs) return (torch.round(torch.flatten(predicted)) == torch.DoubleTensor(labels)).sum().item()/len(circs)

print('Final test accuracy: {}'.format(accuracy(test_pair_circuits, test_labels)))

KeyError Traceback (most recent call last) in <cell line: 1>() ----> 1 print('Final test accuracy: {}'.format(accuracy(test_pair_circuits, test_labels)))

3 frames /usr/local/lib/python3.10/dist-packages/lambeq/training/pennylane_model.py in get_diagram_output(self, diagrams) 178 circuit_evals = [] 179 for d in diagrams: --> 180 p_circ = self.circuit_map[d] 181 p_circ.initialise_concrete_params(self.symbol_weight_map) 182 circuit_evals.append(p_circ.eval())

KeyError: quantum.circuit.Circuit(inside=(rigid.Layer(quantum.circuit.Ty(), quantum.gates.Ket('Ket(0)', quantum.circuit.Ty(), quantum.circuit.Ty(quantum.circuit.Qudit(2))), quantum.circuit.Ty()), rigid.Layer(quantum.circuit.Ty(), quantum.gates.Rx(mann_0), quantum.circuit.Ty()), rigid.Layer(quantum.circuit.Ty(), quantum.gates.Rz(man__n_1), quantum.circuit.Ty()), rigid.Layer(quantum.circuit.Ty(), quantum.gates.Rx(mann_2), quantum.circuit.Ty()), rigid.Layer(quantum.circuit.Ty(quantum.circuit.Qudit(2)), quantum.gates.Ket('Ket(0, 0, 0)', quantum.circuit.Ty(), quantum.circuit.Ty(quantum.circuit.Qudit(2), quantum.circuit.Qudit(2), quantum.circuit.Qudit(2))), quantum.circuit.Ty()), rigid.Layer(quantum.circuit.Ty(quantum.circuit.Qudit(2)), quantum.gates.QuantumGate('H', quantum.circuit.Ty(quantum.circuit.Qudit(2)), quantum.circuit.Ty(quantum.circuit.Qudit(2)), data=[(0.7071067811865476+0j), (0.7071067811865476+0j), (0.7071067811865476+0j), (-0.7071067811865476+0j)]), quantum.circuit.Ty(quantum.circuit.Qudit(2), quantum.circuit.Qudit(2))), rigid.Layer(quantum.circuit.Ty(quantum.circuit.Qudit(2), quantum.circuit.Qudit(2)), quantum.gates.QuantumGate('H', quantum.circuit.Ty(quantum.circuit.Qudit(2)), quantum.circuit.Ty(quantum.circuit.Qudit(2)), data=[(0.7071067811865476+0j), (0.7071067811865476+0j), (0.7071067811865476+0j), (-0.7071067811865476+0j)]), quantum.circuit.Ty(quantum.circuit.Qudit(2))), rigid.Layer(quantum.circuit.Ty(quantum.circuit.Qudit(2), quantum.circuit....

[albi3ro]

Thanks for opening this issue @mohapatra-amrit .

Can you post a minimal non-working example? We do not currently have enough information to be able to reproduce the error.

[mohapatra-amrit]

I am attaching full code that i am running @albi3ro

# Inputting data
def read_data(filename):
    labels, sentences = [], []
    with open(filename) as f:
        for line in f:
            line = line.split(',')
            labels.append(int(line[2]))
            sentences.append((line[0], line[1]))
    return labels, sentences

train_labels, train_data = read_data('mc_pair_train_data.csv')
dev_labels, dev_data = read_data('mc_pair_dev_data.csv')
test_labels, test_data = read_data('mc_pair_test_data.csv')

result = list(zip(train_data[:10], train_labels[:10]))
for item in result:
    print(item)
# Creating and parameterizing diagrams

train_data_l, train_data_r = zip(*train_data)
train_data_unpaired = list(train_data_l) + list(train_data_r)
dev_data_l, dev_data_r = zip(*dev_data)
dev_data_unpaired = list(dev_data_l) + list(dev_data_r)
test_data_l, test_data_r = zip(*test_data)
test_data_unpaired = list(test_data_l) + list(test_data_r)
from lambeq import BobcatParser

reader = BobcatParser(verbose='text')

raw_train_diagrams = reader.sentences2diagrams(train_data_unpaired)
raw_dev_diagrams = reader.sentences2diagrams(dev_data_unpaired)
raw_test_diagrams = reader.sentences2diagrams(test_data_unpaired)
# Simplifying diagrams
from lambeq import remove_cups 

train_diagrams = [remove_cups(diagram) for diagram in raw_train_diagrams]
dev_diagrams = [remove_cups(diagram) for diagram in raw_dev_diagrams]
test_diagrams = [remove_cups(diagram) for diagram in raw_test_diagrams]
# We can visualise these diagrams using discopy.monoidal.Diagram.draw.

train_diagrams[1].draw()
# Creating circuits
from lambeq import AtomicType, IQPAnsatz

ansatz = IQPAnsatz({AtomicType.NOUN: 1, AtomicType.SENTENCE: 1},
                   n_layers=1, n_single_qubit_params=3)

train_circuits = [ansatz(diagram) for diagram in train_diagrams]
dev_circuits =  [ansatz(diagram) for diagram in dev_diagrams]
test_circuits = [ansatz(diagram) for diagram in test_diagrams]

train_circuits[0].draw(figsize=(6, 8))
 # Hybrid QNLP model
BATCH_SIZE = 50
EPOCHS = 100
SEED = 2  
from torch import nn
from lambeq import PennyLaneModel

# inherit from PennyLaneModel to use the PennyLane circuit evaluation
class XORSentenceModel(PennyLaneModel):
    def __init__(self, **kwargs):
        PennyLaneModel.__init__(self, **kwargs)

        self.xor_net = nn.Sequential(nn.Linear(4, 10),
                                     nn.ReLU(),
                                     nn.Linear(10, 1),
                                     nn.Sigmoid())

    def forward(self, diagram_pairs):
        first_d, second_d = zip(*diagram_pairs)
        # evaluate each circuit and concatenate the results
        evaluated_pairs = torch.cat((self.get_diagram_output(first_d),
                                     self.get_diagram_output(second_d)),
                                    dim=1)
        evaluated_pairs = 2 * (evaluated_pairs - 0.5)
        # pass the concatenated results through a simple neural network
        return self.xor_net(evaluated_pairs)
# Creating paired dataset
def make_pair_data(diagrams):
    pair_diags = list(zip(diagrams[:len(diagrams)//2], diagrams[len(diagrams)//2:]))
    return pair_diags

train_pair_circuits = make_pair_data(train_circuits)
dev_pair_circuits = make_pair_data(dev_circuits)
test_pair_circuits = make_pair_data(test_circuits)
# Initializing the model
from lambeq import Dataset

all_pair_circuits = (train_pair_circuits +
                     dev_pair_circuits +
                     test_pair_circuits)
a, b = zip(*all_pair_circuits)

# initialise our model by passing in the diagrams, so that we have trainable parameters for each token
# model = XORSentenceModel.from_diagrams(a + b, probabilities=True, normalize=True)
all_diagrams = a + b  # where a and b are from train, dev, and test pair circuits
model = XORSentenceModel.from_diagrams(all_diagrams, probabilities=True, normalize=True)

model.initialise_weights()
model = model.double()

# initialise datasets and optimizers as in PyTorch
train_pair_dataset = Dataset(train_pair_circuits,
                             train_labels,
                             batch_size=BATCH_SIZE)

optimizer = torch.optim.Adam(model.parameters(), lr=0.1)
# Training and accuracies
def accuracy(circs, labels):
    predicted = model(circs)
    return (torch.round(torch.flatten(predicted)) ==
            torch.DoubleTensor(labels)).sum().item()/len(circs)
 best = {'acc': 0, 'epoch': 0}

for i in range(EPOCHS):
    epoch_loss = 0
    for circuits, labels in train_pair_dataset:
        optimizer.zero_grad()
        predicted = model(circuits)
        # use BCELoss as our outputs are probabilities, and labels are binary
        loss = torch.nn.functional.binary_cross_entropy(
            torch.flatten(predicted), torch.DoubleTensor(labels))
        epoch_loss += loss.item()
        loss.backward()
        optimizer.step()

    # evaluate on dev set every 5 epochs
    # save the model if it's the best so far
    # stop training if the model hasn't improved for 10 epochs
    if i % 5 == 0:
        dev_acc = accuracy(dev_pair_circuits, dev_labels)

        print('Epoch: {}'.format(i))
        print('Train loss: {}'.format(epoch_loss))
        print('Dev acc: {}'.format(dev_acc))

        if dev_acc > best['acc']:
            best['acc'] = dev_acc
            best['epoch'] = i
            model.save('xor_model.lt')
        elif i - best['epoch'] >= 10:
            print('Early stopping')
            break

# load the best performing iteration of the model on the dev set
if best['acc'] > accuracy(dev_pair_circuits, dev_labels):
    model.load('xor_model.lt')
    model = model.double()
print('Final test accuracy: {}'.format(accuracy(test_pair_circuits, test_labels)))     
# Analysing the internal representations of the model
xor_labels = [[1, 0, 1, 0], [0, 1, 0, 1], [1, 0, 0, 1], [0, 1, 1, 0]]
# the first two entries correspond to the same label for both sentences,
# the last two to different labels
xor_tensors = torch.tensor(xor_labels).double()

model.xor_net(xor_tensors).detach().numpy()      
# cooking sentence
print(test_data[0][0],test_data[0][1])

p_circ = test_pair_circuits[0][0].to_pennylane(probabilities=True)
symbol_weight_map = dict(zip(model.symbols, model.weights))
p_circ.initialise_concrete_params(symbol_weight_map)
unnorm = p_circ.eval().detach().numpy()

print(unnorm / np.sum(unnorm))
res = unnorm / np.sum(unnorm)
if(res[0]>res[1]):
  print("Both Sentences are of different topic")
else:
  print("Both Sentences are of same topic")    

[mudit2812]

Hi @mohapatra-amrit ! Could you provide a smaller example that we can use to debug?

I would also suggest opening an issue on the lambeq repository as PennyLane is being accessed through their API.

[mohapatra-amrit]

@mudit2812 getting key error in this line print('Final test accuracy: {}'.format(accuracy(test_pair_circuits, test_labels)))

Accuracy function :

def accuracy(circs, labels):
    predicted = model(circs)
    return (torch.round(torch.flatten(predicted)) ==
            torch.DoubleTensor(labels)).sum().item()/len(circs)

[isaacdevlugt]

Hey @mohapatra-amrit! We're unable to replicate the issue you're facing. As @mudit2812 suggested, it would be a good idea to open an issue on the lambeq repository to see if they have anything to say. Feel free to link this issue with the issue you make on that repository so that we can have a nice paper trail 👍

[Alex-Preciado]

Hi @mohapatra-amrit 👋🏼 . We have not been able to replicate this issue and there hasn't been any activity on it for a significant period of time. As we strive to keep our GitHub repository organized and focused, we are now closing it with this comment. If you believe the issue is still relevant or if you have additional information to provide, please feel free to reopen it, and we'll be happy to continue the discussion 🚀