Multithread

[arnaudbore]

Hello COMMIT Team,

I'm updating scilpy (This PR) and I try to understand how the multithreading works with commit and make sure that when I ask for a specific number of threads it does what it is suppose to do.

I'm testing everything using this quick nextflow pipeline. Here is the command line and script I used test_commit.tar.gz:

nextflow_21.10.6 ~/p/scil/s/noddi_flow/main_commit.nf --input ../i_noddi_flow/ -with-report report.html --nb_threads 8

Then I check the report.html

If you change nb_threads (4 or 8) the report is very confusing. check reports.tar.gz

I tried also to run directly the python script with this command line:

scil_tractogram_commit.py tracking.trk dwi.nii.gz bval bvec     results_bzs/ --tol 30 --nbr_dir 500     --nbr_iter 500 --in_peaks peaks.nii.gz --in_tracking_mask brain_mask.nii.gz     --perp_diff 1.19E-3 0.85E-3 0.51E-3 0.17E-3     --iso_diff 1.7E-3 3.0E-3     --processes 8 --keep_whole_tractogram

Still quite understand why It does not use all the processes I allow him to use.

Thank you in advance for your help Arnaud

[arnaudbore]

@fullbat @daducci Any ideas ?

[daducci]

Hi Arnaud,

We are trying to understand the issue. I have some questions for you.

• To properly interpret your reports, shall we read the values reported as "Run times" or "CPU-Hours"? They seem to go in opposite directions and I don't have experience with Nextflow reports. Sorry!
• To understand where a possible bottleneck is, would it be possible to split the time profiling into two parts? Specifically, we would like to know the execution time (with different threads of course) only when running the function "tck2dictionary.run()" and when running the fit. That would help us a lot.