Closed xand-stapleton closed 1 year ago
You need to pass n_output=3
when instantiating your object using the FIM helper. That way, the generator will expect every minibatch example to produce an output of size 3.
Thanks for your quick reply! Unfortunately that doesn't change the error, except the 1 becomes a 3:
RuntimeError: shape '[9, 3]' is invalid for input of size 3
I think this comes from the fact that you are using a single example, instead of a minibatch of several examples. NNGeometry was designed to work with datasets with many examples.
Update: Yep, that's the problem, making the following change solves it:
class TrivialDataset(Dataset):
def __init__(self):
self.data = torch.arange(9, dtype=torch.float32).view(1,1,9)
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
Thanks! :)
I'm trying to use the latest git release of NNGeometry's FIM to find the Fisher metric of my trivial model. As a stupidly basic example which recreates my problem, I create a model which has a single Linear layer, a single training sample, and solves the matrix equation Ax=b, where A is a 3x3 matrix, whilst x, b are 3x1 col. vectors.
Here's my code (it's not meant for anything functional, it's just to replicate my problem):
Now I create a simple dataloader with that single training sample in (just as a proof of concept):
Now if I try to find the FIM:
There's a runtime error:
I think this comes about because FIM is trying to reshape the output based on the input size. Is this correct?
Thanks