scverse / scvi-tools

Deep probabilistic analysis of single-cell and spatial omics data
http://scvi-tools.org/
BSD 3-Clause "New" or "Revised" License
1.26k stars 357 forks source link

RuntimeError: cannot pin 'torch.cuda.FloatTensor' only dense CPU tensors can be pinned #1713

Closed vinsilico closed 2 years ago

vinsilico commented 2 years ago

The tutorial.py stops during trining the model. I was trying with dataset15 from the example datasets.

CUDA_VISIBLE_DEVICES=7 python3 Tutorial.py

[ERROR HERE]

['Acta2', 'Aldoc', 'Anln', 'Apln', 'Bmp4', 'Cnr1', 'Cpne5', 'Crh', 'Crhbp', 'Ctps', 'Flt1', 'Foxj1', 'Gad2', 'Gfap', 'Hexb', 'Itpr2', 'Kcnip2', 'Lamp5', 'Mfge8', 'Mrc1', 'Pdgfra', 'Plp1', 'Pthlh', 'Rorb', 'Serpinf1', 'Slc32a1', 'Sox10', 'Syt6', 'Tbr1', 'Tmem2', 'Ttr', 'Vip', 'Vtn'] number of cells and genes in the matrix: (5613, 30527) Setting up for reconstruction ... done ( 21.3 seconds ) It. |Err

0|3.830581e-04|

Loading required package: cowplot Registered S3 methods overwritten by 'ggplot2': method from [.quosures rlang c.quosures rlang print.quosures rlang Loading required package: Matrix Loading required package: patchwork

Attaching package: ‘patchwork’

The following object is masked from ‘package:cowplot’:

align_plots

Read 33 items [1] 33 3405 |======================================================================| 100% Finished in 31.61684 secs, 30 iterations. Max iterations: 30. Convergence loss: 4.218993e-06. Best results with seed 1. Warning: This function will discard the raw data previously stored in the liger object and replace the raw.data slot with the imputed data.

Imputing given query datasets Reference dataset: SMSC_RNA Query datasets: SMSC_FISH Registered S3 methods overwritten by 'ggplot2': method from [.quosures rlang c.quosures rlang print.quosures rlang Read 33 items [1] 33 3405 Running CCA Merging objects Finding neighborhoods Finding anchors Found 4408 anchors Running PCA on query dataset Warning in irlba(A = t(x = object), nv = npcs, ...) : You're computing too large a percentage of total singular values, use a standard svd instead. Finding integration vectors Finding integration vector weights 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Transfering 30527 features onto reference data [1] "Gm9318" "LOC105246325" "Abi1" "Rnf213"
[5] "Cops2" "LOC105246475" "Gm6555" "Gm36857"
[9] "Pgpep1l" "Cpsf4" "Prdx6b" "Bco2"
[13] "D830035M03Rik" "Gm31145" "A930024E05Rik" "P2rx1"
[17] "Gm26851" "Epdr1" "Gm8995" "Spsb3"
['Acta2', 'Aldoc', 'Anln', 'Apln', 'Bmp4', 'Cnr1', 'Cpne5', 'Crh', 'Crhbp', 'Ctps', 'Flt1', 'Foxj1', 'Gad2', 'Gfap', 'Hexb', 'Itpr2', 'Kcnip2', 'Lamp5', 'Mfge8', 'Mrc1', 'Pdgfra', 'Plp1', 'Pthlh', 'Rorb', 'Serpinf1', 'Slc32a1', 'Sox10', 'Syt6', 'Tbr1', 'Tmem2', 'Ttr', 'Vip', 'Vtn'] Trying to set attribute .obs of view, copying. Trying to set attribute .obs of view, copying. INFO No batch_key inputted, assuming all cells are same batch
INFO No label_key inputted, assuming all cells have same label
INFO Using data from adata.X
INFO Computing library size prior per batch
INFO Successfully registered anndata object containing 3405 cells, 33 vars, 1 batches, 1 labels, and 0 proteins. Also registered 0 extra
categorical covariates and 0 extra continuous covariates.
INFO Please do not further modify adata until model is trained.
INFO No batch_key inputted, assuming all cells are same batch
INFO No label_key inputted, assuming all cells have same label
INFO Using data from adata.X
INFO Computing library size prior per batch
INFO Successfully registered anndata object containing 5613 cells, 53 vars, 1 batches, 1 labels, and 0 proteins. Also registered 0 extra
categorical covariates and 0 extra continuous covariates.
INFO Please do not further modify adata until model is trained.
INFO Training for 200 epochs.
Traceback (most recent call last): File "Tutorial.py", line 119, in Result = test.Imputing(Methods) File "/swiftcache/x/SpatialBenchmarking/Benchmarking/SpatialGenes.py", line 290, in Imputing result_GimVI = self.gimVI_impute() File "/swiftcache/x/SpatialBenchmarking/Benchmarking/SpatialGenes.py", line 151, in gimVI_impute model.train(200) File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/scvi/model/gimvi.py", line 178, in train self.trainer.train(n_epochs=n_epochs, *train_fun_kwargs) File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/scvi/core/trainers/trainer.py", line 183, in train self.compute_metrics() File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(args, **kwargs) File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/scvi/core/trainers/trainer.py", line 150, in compute_metrics result = getattr(scdl, metric)() File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/scvi/core/trainers/jvae_trainer.py", line 25, in elbo elbo = compute_elbo(self.model, self, mode=self.mode) File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/scvi/core/_log_likelihood.py", line 24, in compute_elbo for i_batch, tensors in enumerate(data_loader): File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 563, in _next_data data = _utils.pin_memory.pin_memory(data) File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/utils/data/_utils/pin_memory.py", line 54, in pin_memory return {k: pin_memory(sample) for k, sample in data.items()} File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/utils/data/_utils/pin_memory.py", line 54, in return {k: pin_memory(sample) for k, sample in data.items()} File "/home/x/.conda/envs/Benchmarking/lib/python3.6/site-packages/torch/utils/data/_utils/pin_memory.py", line 50, in pin_memory return data.pin_memory() RuntimeError: cannot pin 'torch.cuda.FloatTensor' only dense CPU tensors can be pinned

Versions:

VERSION gimVI

adamgayoso commented 2 years ago

Can you try using python >= 3.8?

adamgayoso commented 2 years ago

We can't help with this issue without more information, so closing for now.