kuleshov-group / caduceus

Bi-Directional Equivariant Long-Range DNA Sequence Modeling
Apache License 2.0
137 stars 14 forks source link

num_workers=1 warning #29

Closed leannmlindsey closed 1 month ago

leannmlindsey commented 2 months ago

I keep getting this warning:

/uufs/chpc.utah.edu/common/home/u1323098/software/pkg/miniconda3/envs/CADUCEUS_3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:224: PossibleUserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 128 which is the number of cpus on this machine) in theDataLoader` init to improve performance.

This seems to be originating from the genomics.py script where it is set as num_workers=1

Is there a reason that you have set it at 1? can this be increased?

yair-schiff commented 2 months ago

You should be able to set number of workers with this param: loader.num_workers

leannmlindsey commented 1 month ago

Thanks.