search

snap-stanford / SATURN

MIT License
108 stars 17 forks source link

RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu) #57

Closed matejasoretic closed 6 months ago

matejasoretic commented 6 months ago

I ran a successful integration with SATURN, however, I encountered the following error when I ran the same command that ran successfully, the only difference being I included --balance_pretrain and --balance_species_cells 

Saved centroids to /path/to//all_species_centroids.pkl
Pretraining...
  0%|          | 0/200 [00:03<?, ?it/s]
Traceback (most recent call last):
  File " /path/to/train-saturn.py", line 1064, in <module>
    trainer(args)
  File " /path/to/train-saturn.py", line 654, in trainer
    pretrain_model = pretrain_saturn(pretrain_model, pretrain_loader, optim_pretrain,
  File " /path/to/train-saturn.py", line 231, in pretrain_saturn
    batch_weights = label_weights[labels].to(device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

What could be the cause of this error?

matejasoretic commented 6 months ago

Right before line 231, I added this line: label_weights = label_weights.to(device)

Now it works; I think this issue was possibly something to do with the newer version of cuda on our server.