MichaelHiller
commented
4 months ago If I understand it right, jobs are running, just very slow. There is no multithreading. Rather the data is split into chunks and each chunk can run in parallel, either on a compute cluster or a single node (where you can then use all its cores).
Regarding speed, pls check if you repeatModeled and masked both genomes (hg38 should be fine though).
@kirilenkobm Can you comment on the --executor_queuesize parameter?
Hi To obtain the prequel input file for TOGA, I attempted the process based on the information provided, but encountered some difficulties. I'm unsure how to proceed and would appreciate any suggestions or guidance you may have. Thank you for your assistance. Here are my command and the erro log: ./make_chains.py hg38 Ea /home/make_lastz_chains/genome/hg38.2bit /home/make_lastz_chains/genome/Ea.2bit --pd /home/Ea -f --chaining_memory 16 I use --executor local instead of --executor slurm, is there any parameter to speed it up, it's too slow current output : 1、 N E X T F L O W ~ version 21.10.6 Launching /home/make_lastz_chains/parallelization/execute_joblist.nf [intergalactic_khorana] - revision: 0483b29723 executor > local (5) [5f/78dfbd] process > execute_jobs (2) [ 0%] 4 of 105150 2、 N E X T F L O W ~ version 21.10.6 Launching /home/make_lastz_chain/make_lastz_chains/parallelization/execute_joblist.nf [wise_austin] - revision: 0483b29723 executor > local (207) [78/011ee5] process > execute_jobs (223) [ 9%] 196 of 2010 In addition, I noticed in the previous Q&As that there is a parameter "--executor_queuesize 8" that can make it multi-threaded, but it seems to no longer exist in the current version, whether it has been replaced by "--cluster_queue 8", My running speed is about 5% of the number of jobs finished a day, do you have any suggestions for this, thanks a lot. best wishes