doraadong / UNIFAN

Unsupervised cell functional annotation for single-cell RNA-Seq
MIT License
22 stars 3 forks source link

TypeError: can't convert cuda:0 device type tensor to numpy. #4

Open auesro opened 2 years ago

auesro commented 2 years ago

Dear @doraadong

I have found an error when running the cluster annotations notebook

At the line weights = model_dict['state_dict_2']['decoder.predictor.0.weight'].detach().numpy(), python throws:

Traceback (most recent call last):

  File "/tmp/ipykernel_286232/2969313530.py", line 1, in <module>
    weights = model_dict['state_dict_2']['decoder.predictor.0.weight'].detach().numpy()

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

I believe I solved it by doing:

weights = model_dict['state_dict_2']['decoder.predictor.0.weight'].cpu()
sub_weights = weights[unique_clusters, :].numpy()

The notebook then runs without problems but wanted to make sure thats the correct solution.

Cheers,

A

P.S.: Just in case, I add here my environment setup:

# packages in environment at /home/auesro/mambaforge/envs/Unifan:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
alsa-lib                  1.2.6.1              h7f98852_0    conda-forge
anndata                   0.8.0                    pypi_0    pypi
arpack                    3.7.0                hdefa2d7_2    conda-forge
arrow                     1.2.2              pyhd8ed1ab_0    conda-forge
astroid                   2.11.6           py37h89c1867_0    conda-forge
atomicwrites              1.4.0              pyh9f0ad1d_0    conda-forge
attr                      2.5.1                h166bdaf_0    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
autopep8                  1.6.0              pyhd8ed1ab_1    conda-forge
babel                     2.10.3             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.11.1             pyha770c72_0    conda-forge
binaryornot               0.4.4                      py_1    conda-forge
black                     22.6.0             pyhd8ed1ab_0    conda-forge
blas                      2.115                       mkl    conda-forge
blas-devel                3.9.0            15_linux64_mkl    conda-forge
bleach                    5.0.1              pyhd8ed1ab_0    conda-forge
brotlipy                  0.7.0           py37h540881e_1004    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.6.15            ha878542_0    conda-forge
certifi                   2022.6.15        py37h89c1867_0    conda-forge
cffi                      1.15.1           py37h43b0acd_0    conda-forge
chardet                   5.0.0            py37h89c1867_0    conda-forge
charset-normalizer        2.1.0              pyhd8ed1ab_0    conda-forge
click                     8.1.3            py37h89c1867_0    conda-forge
cloudpickle               2.1.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.5              pyhd8ed1ab_0    conda-forge
cookiecutter              2.1.1              pyh6c4a22f_0    conda-forge
cryptography              37.0.4           py37h38fbfac_0    conda-forge
cudatoolkit               11.3.1              h9edb442_10    conda-forge
cycler                    0.11.0                   pypi_0    pypi
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
debugpy                   1.6.0            py37hd23a5d3_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
diff-match-patch          20200713           pyh9f0ad1d_0    conda-forge
dill                      0.3.5.1            pyhd8ed1ab_0    conda-forge
docutils                  0.17.1           py37h89c1867_2    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
expat                     2.4.8                h27087fc_0    conda-forge
ffmpeg                    4.3                  hf484d3e_0    pytorch
fftw                      3.3.10          nompi_h77c792f_102    conda-forge
flake8                    4.0.1              pyhd8ed1ab_2    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.0               h8e229c2_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.34.4                   pypi_0    pypi
freetype                  2.10.4               h0708190_1    conda-forge
gettext                   0.19.8.1          h73d1719_1008    conda-forge
giflib                    5.2.1                h36c2ea0_2    conda-forge
glib                      2.70.2               h780b84a_4    conda-forge
glib-tools                2.70.2               h780b84a_4    conda-forge
glpk                      4.65              h9202a9a_1004    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gnutls                    3.6.13               h85f3911_1    conda-forge
gst-plugins-base          1.20.3               hf6a322e_0    conda-forge
gstreamer                 1.20.3               hd4edc92_0    conda-forge
h5py                      3.7.0                    pypi_0    pypi
icu                       70.1                 h27087fc_0    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
igraph                    0.9.9                h026ac8f_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        4.2.0            py37h89c1867_0    conda-forge
importlib_metadata        4.2.0                hd8ed1ab_0    conda-forge
importlib_resources       5.8.0              pyhd8ed1ab_0    conda-forge
inflection                0.5.1              pyh9f0ad1d_0    conda-forge
intervaltree              3.0.2                      py_0    conda-forge
ipykernel                 6.15.0             pyh210e3f2_0    conda-forge
ipython                   7.33.0           py37h89c1867_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
isort                     5.10.1             pyhd8ed1ab_0    conda-forge
jack                      1.9.18            h8c3723f_1002    conda-forge
jedi                      0.18.1           py37h89c1867_1    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jellyfish                 0.9.0            py37h540881e_1    conda-forge
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
jinja2-time               0.2.0              pyhd8ed1ab_3    conda-forge
joblib                    1.1.0                    pypi_0    pypi
jpeg                      9e                   h166bdaf_2    conda-forge
jsonschema                4.6.1              pyhd8ed1ab_0    conda-forge
jupyter_client            7.3.4              pyhd8ed1ab_0    conda-forge
jupyter_core              4.10.0           py37h89c1867_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
keyring                   23.6.0           py37h89c1867_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.3                    pypi_0    pypi
krb5                      1.19.3               h3790be6_0    conda-forge
lame                      3.100             h7f98852_1001    conda-forge
lazy-object-proxy         1.7.1            py37h540881e_1    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
leidenalg                 0.8.10           py37hd23a5d3_0    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libblas                   3.9.0            15_linux64_mkl    conda-forge
libcap                    2.64                 ha37c62d_0    conda-forge
libcblas                  3.9.0            15_linux64_mkl    conda-forge
libclang                  14.0.6          default_h2e3cab8_0    conda-forge
libclang13                14.0.6          default_h3a83d3e_0    conda-forge
libcups                   2.3.3                hf5a7f15_1    conda-forge
libdb                     6.2.32               h9c3ff4c_0    conda-forge
libdeflate                1.12                 h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.3.4                h27087fc_0    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libglib                   2.70.2               h174f98d_4    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0            15_linux64_mkl    conda-forge
liblapacke                3.9.0            15_linux64_mkl    conda-forge
libllvm14                 14.0.6               he0ac6c6_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.20          pthreads_h78a6416_0    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpng                    1.6.37               h753d276_3    conda-forge
libpq                     14.4                 hd77ab85_0    conda-forge
libsndfile                1.0.31               h9c3ff4c_1    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libtiff                   4.4.0                hc85c160_1    conda-forge
libtool                   2.4.6             h9c3ff4c_1008    conda-forge
libudev1                  249                  h166bdaf_4    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libuv                     1.43.0               h7f98852_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libwebp                   1.2.2                h3452ae3_0    conda-forge
libwebp-base              1.2.2                h7f98852_1    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.14               h22db469_3    conda-forge
libzlib                   1.2.12               h166bdaf_1    conda-forge
llvm-openmp               14.0.4               he0ac6c6_0    conda-forge
llvmlite                  0.38.1                   pypi_0    pypi
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
markupsafe                2.1.1            py37h540881e_1    conda-forge
matplotlib                3.5.2                    pypi_0    pypi
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
metis                     5.1.0             h58526e2_1006    conda-forge
mistune                   0.8.4           py37h5e8e339_1005    conda-forge
mkl                       2022.1.0           h84fe81f_915    conda-forge
mkl-devel                 2022.1.0           ha770c72_916    conda-forge
mkl-include               2022.1.0           h84fe81f_915    conda-forge
mpfr                      4.1.0                h9202a9a_1    conda-forge
mypy_extensions           0.4.3            py37h89c1867_5    conda-forge
mysql-common              8.0.29               haf5c9bc_1    conda-forge
mysql-libs                8.0.29               h28c427c_1    conda-forge
natsort                   8.1.0                    pypi_0    pypi
nbclient                  0.6.6              pyhd8ed1ab_0    conda-forge
nbconvert                 6.5.0              pyhd8ed1ab_0    conda-forge
nbconvert-core            6.5.0              pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          6.5.0              pyhd8ed1ab_0    conda-forge
nbformat                  5.4.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nest-asyncio              1.5.5              pyhd8ed1ab_0    conda-forge
nettle                    3.6                  he412f7d_0    conda-forge
networkx                  2.6.3                    pypi_0    pypi
nspr                      4.32                 h9c3ff4c_1    conda-forge
nss                       3.78                 h2350873_0    conda-forge
numba                     0.55.2                   pypi_0    pypi
numpy                     1.21.6           py37h976b520_0    conda-forge
numpydoc                  1.4.0              pyhd8ed1ab_0    conda-forge
openh264                  2.1.1                h780b84a_0    conda-forge
openjpeg                  2.4.0                hb52868f_1    conda-forge
openssl                   1.1.1q               h166bdaf_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.3.5                    pypi_0    pypi
pandoc                    2.18                 ha770c72_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pathspec                  0.9.0              pyhd8ed1ab_0    conda-forge
patsy                     0.5.2                    pypi_0    pypi
pcre                      8.45                 h9c3ff4c_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.2.0            py37h44f0d7a_0    conda-forge
pip                       22.1.2             pyhd8ed1ab_0    conda-forge
platformdirs              2.5.1              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0            py37h89c1867_3    conda-forge
portaudio                 19.6.0               h57a0ea0_5    conda-forge
prompt-toolkit            3.0.30             pyha770c72_0    conda-forge
psutil                    5.9.1            py37h540881e_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pulseaudio                14.0                 h7f54b18_8    conda-forge
pycodestyle               2.8.0              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydocstyle                6.1.1              pyhd8ed1ab_0    conda-forge
pyflakes                  2.4.0              pyhd8ed1ab_0    conda-forge
pygments                  2.12.0             pyhd8ed1ab_0    conda-forge
pylint                    2.14.4             pyhd8ed1ab_0    conda-forge
pyls-spyder               0.4.0              pyhd8ed1ab_0    conda-forge
pynndescent               0.5.7                    pypi_0    pypi
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.4           py37hf30b843_1    conda-forge
pyqt5-sip                 12.9.0           py37hd23a5d3_1    conda-forge
pyqtwebengine             5.15.4           py37hf30b843_1    conda-forge
pyrsistent                0.18.1           py37h540881e_1    conda-forge
pysocks                   1.7.1            py37h89c1867_5    conda-forge
python                    3.7.12          hb7a2778_100_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.15.3             pyhd8ed1ab_0    conda-forge
python-igraph             0.9.11           py37h87e8262_0    conda-forge
python-lsp-black          1.2.1              pyhd8ed1ab_0    conda-forge
python-lsp-jsonrpc        1.0.0              pyhd8ed1ab_0    conda-forge
python-lsp-server         1.4.1              pyhd8ed1ab_1    conda-forge
python-slugify            6.1.2              pyhd8ed1ab_0    conda-forge
python_abi                3.7                     2_cp37m    conda-forge
pytoolconfig              1.2.1              pyhd8ed1ab_0    conda-forge
pytorch                   1.10.1          py3.7_cuda11.3_cudnn8.2.0_0    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2022.1             pyhd8ed1ab_0    conda-forge
pyxdg                     0.28               pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py37h540881e_4    conda-forge
pyzmq                     23.2.0           py37h0c0c2a8_0    conda-forge
qdarkstyle                3.0.3              pyhd8ed1ab_0    conda-forge
qstylizer                 0.2.1              pyhd8ed1ab_0    conda-forge
qt-main                   5.15.4               ha5833f6_2    conda-forge
qt-webengine              5.15.4               hcbadb6c_3    conda-forge
qtawesome                 1.1.1              pyhd8ed1ab_0    conda-forge
qtconsole                 5.3.1              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.3.1              pyha770c72_0    conda-forge
qtpy                      2.1.0              pyhd8ed1ab_0    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_0    conda-forge
rope                      1.2.0              pyhd8ed1ab_0    conda-forge
rtree                     1.0.0            py37h0b55af0_1    conda-forge
scanpy                    1.9.1                    pypi_0    pypi
scikit-learn              1.0.2                    pypi_0    pypi
scikit-misc               0.1.4            py37h7bf8d0b_3    conda-forge
scipy                     1.7.3                    pypi_0    pypi
seaborn                   0.11.2                   pypi_0    pypi
secretstorage             3.3.2            py37h89c1867_1    conda-forge
session-info              1.0.0                    pypi_0    pypi
setuptools                63.1.0           py37h89c1867_0    conda-forge
sip                       6.5.1            py37hcd2ae1e_2    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.1              pyhd8ed1ab_0    conda-forge
sphinx                    4.3.2              pyh6c4a22f_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
spyder                    5.3.1            py37h89c1867_0    conda-forge
spyder-kernels            2.3.1            py37h89c1867_0    conda-forge
sqlite                    3.39.0               h4ff8645_0    conda-forge
statsmodels               0.13.2                   pypi_0    pypi
stdlib-list               0.8.0                    pypi_0    pypi
suitesparse               5.10.1               h9e50725_1    conda-forge
tbb                       2021.5.0             h924138e_1    conda-forge
text-unidecode            1.3                        py_0    conda-forge
textdistance              4.3.0              pyhd8ed1ab_0    conda-forge
texttable                 1.6.4              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.1.0                    pypi_0    pypi
three-merge               0.1.1              pyh9f0ad1d_0    conda-forge
tinycss2                  1.1.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.11.1             pyha770c72_0    conda-forge
torchaudio                0.10.1               py37_cu113    pytorch
torchvision               0.11.2               py37_cu113    pytorch
tornado                   6.2              py37h540881e_0    conda-forge
tqdm                      4.64.0                   pypi_0    pypi
traitlets                 5.3.0              pyhd8ed1ab_0    conda-forge
typed-ast                 1.5.4            py37h540881e_0    conda-forge
typing                    3.10.0.0           pyhd8ed1ab_0    conda-forge
typing-extensions         4.3.0                hd8ed1ab_0    conda-forge
typing_extensions         4.3.0              pyha770c72_0    conda-forge
ujson                     5.3.0            py37hd23a5d3_0    conda-forge
umap-learn                0.5.3                    pypi_0    pypi
unidecode                 1.3.4              pyhd8ed1ab_0    conda-forge
unifan                    1.0.0                    pypi_0    pypi
urllib3                   1.26.9             pyhd8ed1ab_0    conda-forge
watchdog                  2.1.9            py37h89c1867_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
wrapt                     1.14.1           py37h540881e_0    conda-forge
wurlitzer                 3.0.2              pyhd8ed1ab_2    conda-forge
xcb-util                  0.4.0                h166bdaf_0    conda-forge
xcb-util-image            0.4.0                h166bdaf_0    conda-forge
xcb-util-keysyms          0.4.0                h166bdaf_0    conda-forge
xcb-util-renderutil       0.3.9                h166bdaf_0    conda-forge
xcb-util-wm               0.4.1                h166bdaf_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yapf                      0.32.0             pyhd8ed1ab_0    conda-forge
zeromq                    4.3.4                h9c3ff4c_1    conda-forge
zipp                      3.8.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.12               h166bdaf_1    conda-forge
zstd                      1.5.2                h8a70e8d_2    conda-forge
doraadong commented 2 years ago

Hi, thanks for reporting this! Yes, your correction should be fine. Or we can just add ".cpu()" and having:

model_dict['state_dict_2']['decoder.predictor.0.weight'].detach().cpu().numpy()

I just updated the corresponding line in the tutorial.

auesro commented 2 years ago

No problem, Dora!

I have a couple of questions, I will add them here but if you prefer me to post them somewhere else, let me know: -In the tutorial, you use human genesets (c2,c5, and c8) to cluster a mouse dataset. Would it work better to use the mouse genesets? Or it doesnt matter much? -Can we use the updated genesets (v2022.1) or Unifan is designed to work only with the ones provided (v7.4)? -Can we add other genesets such as c3.mir or c3.tft to aid in the clustering?

Thanks!

A

doraadong commented 2 years ago

Hi, UNIFAN should be compatible with any gene sets. We use getGeneSetMatrix function in the utils.py to process the gene sets input. You may take a look at the function and our example gene sets (including the tf-DNA interaction file) to see if this function can also be applied to process the gene sets that you are interested in. Let me know if modifications are needed and we will consider update the function accordingly.

Regarding your question on mouse or human gene sets, we haven't explored if applying mouse (human) gene sets to mouse (human) tissues works better. It is an interesting question and I would love to hear more from you if you decide to explore this.