search

gao-lab / GLUE

Graph-linked unified embedding for single-cell multi-omics data integration
MIT License
382 stars 56 forks source link

Early stopping during model training #85

Closed YangLi-Bio closed 1 year ago

YangLi-Bio commented 1 year ago

Dear developer,

I succeeded to perform data preprocessing using GLUE. However, when I tried to train the model via scglue.models.fit_SCGLUE using the example data provided by the official website of GLUE, I came across "Early stopping" and got a ValueError. Details are as follows.

Command where error occurrs

glue = scglue.models.fit_SCGLUE( {"rna": rna, "atac": atac}, guidance_hvf, fit_kws = {"directory": "glue"} )

Error messages

[INFO] SCGLUETrainer: [Epoch 60] train={'g_nll': 0.384, 'g_kl': 0.004, 'g_elbo': 0.389, 'x_rna_nll': 0.147, 'x_rna_kl': 0.001, 'x_rna_elbo': 0.148, 'x_atac_nll': 0.039, 'x_atac_kl': 0.0, 'x_atac_elbo': 0.039, 'dsc_loss': 0.691, 'vae_loss': 0.202, 'gen_loss': 0.168}, val={'g_nll': 0.384, 'g_kl': 0.004, 'g_elbo': 0.389, 'x_rna_nll': 0.149, 'x_rna_kl': 0.001, 'x_rna_elbo': 0.15, 'x_atac_nll': 0.039, 'x_atac_kl': 0.0, 'x_atac_elbo': 0.039, 'dsc_loss': 0.699, 'vae_loss': 0.205, 'gen_loss': 0.17}, 22.5s elapsed 2023-07-19 12:15:32,939 ignite.handlers.early_stopping.EarlyStopping INFO: EarlyStopping: Stop training [INFO] EarlyStopping: Restoring checkpoint "56"... Trained GLUE model was saved to /fs/ess/PCON0022/liyang/STREAM-revision/Feasibility/glue-example//glue.dill Traceback (most recent call last): File "/fs/ess/PCON0022/liyang/STREAM-revision/Feasibility/glue/3-GLUE-model-training.py", line 67, in dx = scglue.models.integration_consistency( File "/users/PAS1475/liyang/.conda/envs/scglue/lib/python3.10/site-packages/scglue/models/dx.py", line 49, in integration_consistency adatas = { File "/users/PAS1475/liyang/.conda/envs/scglue/lib/python3.10/site-packages/scglue/models/dx.py", line 50, in k: AnnData( File "/users/PAS1475/liyang/.conda/envs/scglue/lib/python3.10/site-packages/anndata/_core/anndata.py", line 285, in init self._init_as_actual( File "/users/PAS1475/liyang/.conda/envs/scglue/lib/python3.10/site-packages/anndata/_core/anndata.py", line 505, in _init_as_actual self._check_dimensions() File "/users/PAS1475/liyang/.conda/envs/scglue/lib/python3.10/site-packages/anndata/_core/anndata.py", line 1850, in _check_dimensions raise ValueError( ValueError: Variables annot. var must have number of columns of X (28930), but has 27659 rows.

Jeff1995 commented 1 year ago

Hi @YangLi-Bio! Thanks for your interest in GLUE!

According to the traceback, the error is caused by scglue.models.integration_consistency rather than early stopping. It seems that the .var slot of at least one of the AnnData objects is missing rows for some reason.

I rerun the tutorial in a newly created environment with all packages at their latest versions, but could not reproduce this error. Could you check if there was some operation that changed the .var slot and caused removal of some rows?

YangLi-Bio commented 1 year ago

Thank you for your patience and understanding. I find it essential to proceed with upgrading all the modules before re-executing GLUE. Once this task is completed, I will review the results meticulously. Rest assured, I will loop back with you to provide an update at the earliest.

Your cooperation is greatly appreciated. Thank you!

YangLi-Bio commented 1 year ago

Dear Dr. Cao,

This issue has been resolved. The reason for this issue is elementary, i.e., my virtual environment is not built via 'conda'.

Please close this issue. Thank you!