search

yezhengSTAT / scVI-3D

GNU General Public License v3.0
5 stars 2 forks source link

Error when changing --bandMax option to "whole" #7

Closed donalday closed 9 months ago

donalday commented 9 months ago

Hi Ye,

Thank you for your wonderful tool. I have just started using your tool. I have successfully installed everything needed (e.g. scvi-tools=0.14.6). The demo test run completely without problem. However, if I use the same demo dataset but changing the -bandMax option to "whole", the run stops after numerous rounds of training (at the end of this message).

I want to know what is the meaning and impact of bandMax option? Is it always best to use "whole" option? Do you have any hint how to solve the error?

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/user/Data/scVI-3D-gitclone/scripts/scVI-3D.py", line 743, in res = Parallel(n_jobs=coreN,backend='multiprocessing')(delayed(normalize)(bandM, cellInfo, chromSelect, bandDist, nLatent, batchFlag, gpuFlag) for bandM, chromSelect, bandDist in bandMiter) File "/home/user/anaconda3/envs/schic-scvi-3d/lib/python3.8/site-packages/joblib/parallel.py", line 1061, in call self.retrieve() File "/home/user/anaconda3/envs/schic-scvi-3d/lib/python3.8/site-packages/joblib/parallel.py", line 938, in retrieve self._output.extend(job.get(timeout=self.timeout)) File "/home/user/anaconda3/envs/schic-scvi-3d/lib/python3.8/multiprocessing/pool.py", line 771, in get raise self._value ValueError: Dataloader returned 0 length. Please make sure that it returns at least 1 batch

yezhengSTAT commented 9 months ago

Hello, Using whole bands can be risky if the resolution is high. You can imagine that the upper right corner of the Hi-C contact matrix can be very very sparse and it is possible that all the entries in those bands are 0 or only have values in one batch. Therefore, there is no meaning to normalizing or batch-correcting such sparse entries. Therefore, it is better to set an upper bound to avoid handling all zeros in far away off-diagonals.

Hope it helps.

Thanks, Ye

donalday commented 9 months ago

Hi Ye,

Thank you very much. I will try different -b values to see how much difference.