Closed Coder-Liuu closed 1 year ago
I also face this issues sometimes and sometimes my code runs fine.
I believe this issue is related to the empty initialisation of the background embedding tensor in the STALE model class.
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) )
This means the tensor will have whatever values were already present in that memory block. These could be zeros, random numbers, or even nan or inf values, depending on what operations were performed in that memory space previously. This could also explain why the code works sometimes and not others.
One solution is to initialise the background embedding with values as such:
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) ) torch.nn.init.kaiming_uniform_(self.bg_embeddings, nonlinearity='relu')
However I have not checked to see if there was a specific reason that the background embedding is left empty. @sauradip please could you let us know if this solution would be acceptable?
I believe this issue is related to the empty initialisation of the background embedding tensor in the STALE model class.
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) )
This means the tensor will have whatever values were already present in that memory block. These could be zeros, random numbers, or even nan or inf values, depending on what operations were performed in that memory space previously. This could also explain why the code works sometimes and not others.
One solution is to initialise the background embedding with values as such:
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) ) torch.nn.init.kaiming_uniform_(self.bg_embeddings, nonlinearity='relu')
However I have not checked to see if there was a specific reason that the background embedding is left empty. @sauradip please could you let us know if this solution would be acceptable?
Thanks for your advice @ed-fish ! I have tried this solution multi-runs and found it works, zero initialization may be the reason for the error "DivBackward0".
I believe this issue is related to the empty initialisation of the background embedding tensor in the STALE model class.
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) )
This means the tensor will have whatever values were already present in that memory block. These could be zeros, random numbers, or even nan or inf values, depending on what operations were performed in that memory space previously. This could also explain why the code works sometimes and not others.
One solution is to initialise the background embedding with values as such:
self.bg_embeddings = nn.Parameter( torch.empty(1, 512) ) torch.nn.init.kaiming_uniform_(self.bg_embeddings, nonlinearity='relu')
However I have not checked to see if there was a specific reason that the background embedding is left empty. @sauradip please could you let us know if this solution would be acceptable?
Good Spot ! You can initializae anything. Sorry for this error. I will correct it
I downloaded the code and dataset, and modified only anet.yaml, but I still have this problem, can you help me?
My environment and configuration:
Detailed error reporting