canergen
commented
22 hours ago Please try out our recent changes by installing scvi-tools from the main branch. I hope this is fixed there. Those will be released with scvi-tools 1.2
Open raozuming opened 23 hours ago
canergen
commented
22 hours ago Please try out our recent changes by installing scvi-tools from the main branch. I hope this is fixed there. Those will be released with scvi-tools 1.2
raozuming
commented
22 hours ago @canergen Thank you for your reply. Since scvi-tools 1.2 requires python > 3.8, I cannot upgrade the python version. Secondly, I tried to manually update the commit you fixed (https://github.com/scverse/scvi-tools/pull/2632/files) to _trainingplans.py, and the problem still occurs.
mt1022
commented
16 hours ago I am using python 3.12 and have the same problem when training a scvi.model.SCVI model. I tried to install the main branch or the 1.2.x branch and the same error was met.
cmd:
model.train(early_stopping=True, accelerator='cpu') # same error with "mps"error:
File [~/miniforge3/envs/sc/lib/python3.12/site-packages/torch/distributions/distribution.py:70](http://localhost:8890/lab/tree/~/miniforge3/envs/sc/lib/python3.12/site-packages/torch/distributions/distribution.py#line=69), in Distribution.__init__(self, batch_shape, event_shape, validate_args)
68 valid = constraint.check(value)
69 if not valid.all():
---> 70 raise ValueError(
71 f"Expected parameter {param} "
72 f"({type(value).__name__} of shape {tuple(value.shape)}) "
73 f"of distribution {repr(self)} "
74 f"to satisfy the constraint {repr(constraint)}, "
75 f"but found invalid values:\n{value}"
76 )
77 super().__init__()
ValueError: Expected parameter loc (Tensor of shape (128, 10)) of distribution Normal(loc: torch.Size([128, 10]), scale: torch.Size([128, 10])) to satisfy the constraint Real(), but found invalid values:
tensor([[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]], grad_fn=<AddmmBackward0>)session info
-----
anndata 0.10.8
scanpy 1.10.3
-----
CoreFoundation NA
Foundation NA
PIL 10.4.0
PyObjCTools NA
absl NA
anyio NA
appnope 0.1.4
arrow 1.3.0
asttokens NA
attr 24.2.0
attrs 24.2.0
babel 2.14.0
brotli 1.1.0
certifi 2024.08.30
cffi 1.17.1
charset_normalizer 3.3.2
chex 0.1.86
colorama 0.4.6
comm 0.2.2
contextlib2 NA
cycler 0.12.1
cython_runtime NA
dateutil 2.9.0
debugpy 1.8.5
decorator 5.1.1
defusedxml 0.7.1
docrep 0.3.2
etils 1.9.4
executing 2.1.0
fastjsonschema NA
filelock 3.16.1
flax 0.9.0
fqdn NA
fsspec 2024.9.0
gmpy2 2.1.5
google NA
h5py 3.11.0
idna 3.10
importlib_resources NA
ipykernel 6.29.5
isoduration NA
jax 0.4.31
jaxlib 0.4.31
jedi 0.19.1
jinja2 3.1.4
joblib 1.4.2
json5 0.9.25
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema_specifications NA
jupyter_events 0.10.0
jupyter_server 2.14.2
jupyterlab_server 2.27.3
kiwisolver 1.4.7
legacy_api_wrap NA
lightning 2.4.0
lightning_utilities 0.11.7
llvmlite 0.43.0
markupsafe 2.1.5
matplotlib 3.9.2
ml_collections NA
ml_dtypes 0.5.0
mpl_toolkits NA
mpmath 1.3.0
msgpack 1.1.0
mudata 0.3.1
multipledispatch 0.6.0
natsort 8.4.0
nbformat 5.10.4
numba 0.60.0
numpy 1.26.4
numpyro 0.15.3
objc 10.3.1
opt_einsum v3.3.0
optax 0.2.2
overrides NA
packaging 24.1
pandas 2.2.2
parso 0.8.4
patsy 0.5.6
pickleshare 0.7.5
platformdirs 4.3.6
prometheus_client NA
prompt_toolkit 3.0.47
psutil 6.0.0
pure_eval 0.2.3
pycparser 2.22
pydev_ipython NA
pydevconsole NA
pydevd 2.9.5
pydevd_file_utils NA
pydevd_plugins NA
pydevd_tracing NA
pygments 2.18.0
pynndescent 0.5.13
pyparsing 3.1.4
pyro 1.9.1+0a67ddc
pythonjsonlogger NA
pytz 2024.2
referencing NA
requests 2.32.3
rfc3339_validator 0.1.4
rfc3986_validator 0.1.1
rich NA
rpds NA
scipy 1.14.1
scvi 1.1.6
send2trash NA
session_info 1.0.0
six 1.16.0
sklearn 1.5.2
sniffio 1.3.1
socks 1.7.1
sparse 0.15.4
stack_data 0.6.2
statsmodels 0.14.3
sympy 1.13.2
threadpoolctl 3.5.0
toolz 0.12.1
torch 2.4.0
torchgen NA
torchmetrics 1.4.2
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
typing_extensions NA
umap 0.5.6
uri_template NA
urllib3 2.2.3
wcwidth 0.2.13
webcolors 24.8.0
websocket 1.8.0
xarray 2024.9.0
yaml 6.0.2
zmq 26.2.0
zstandard 0.23.0
-----
IPython 8.27.0
jupyter_client 8.6.3
jupyter_core 5.7.2
jupyterlab 4.2.5
-----
Python 3.12.6 | packaged by conda-forge | (main, Sep 11 2024, 04:55:15) [Clang 17.0.6 ]
macOS-14.6.1-arm64-arm-64bit
-----
Session information updated at 2024-09-19 22:09@raozuming We don’t support the use of outdated Python versions. There was an unfortunate stack of activation functions. It is safe to try several runs to get one that succeeds (like 10). If the issue persists, please set up a new environment with a supported Python version 3.10-3.12. @mt1022 It’s a different issue as it’s a different model. Please check out: https://docs.scvi-tools.org/en/latest/faq.html. It is usually a problem with low count cells that give bad gradients. Please share your dataset size, all parts of your code and verify that your AnnData contains counts.
[TEXT HERE]
Versions:
python 3.8 scvi-tools 0.19.0