search

MannLabs / alphapeptdeep

Deep learning framework for proteomics
Apache License 2.0
105 stars 20 forks source link

Multiprocessing causes hanging for library creation; also, multiprocessing flag in yaml is IGNORED #73

Closed gsaxena888 closed 1 year ago

gsaxena888 commented 1 year ago

When trying to run peptdeep library creation on an 8 core linux docker, it hangs. Please see attached logs. Also, when I tried to disable multiprocessing in the yaml, it still continues to use multiprocessing. I've attached the output log, the settings*.yaml file and the input file. My current workaround: I manually edited the pretrained_models.py file to force DISABLING of mutliprocessing, and then it appeared to work.

peptdeep.log settingsLibrary.yaml.gz cyto_all.tsv_forPred.tsv.gz

Ideally, multiprocessing should work etc., but what's really odd is why the multiprocessing flag in the yaml file is ignored etc.

jalew188 commented 1 year ago

I guess the problem is on fork and spawn in linux multiprocessing, I will try to fix this

jalew188 commented 1 year ago

For multiprocessing in yaml, I have found the bug, I will FIX and update all these features in v1.0.0.

jalew188 commented 1 year ago

The latest development branch solved both of the multiprocessing and yaml issues, check https://github.com/MannLabs/alphapeptdeep/blob/development/nbs_tests/test_fasta_lib_cpu_mp.ipynb

jalew188 commented 1 year ago

All these new FIX will be released in v1.0.0.