Multithread

[arnaudbore]

Hello COMMIT Team,

I'm updating scilpy (This PR) and I try to understand how the multithreading works with commit and make sure that when I ask for a specific number of threads it does what it is suppose to do.

I'm testing everything using this quick nextflow pipeline. Here is the command line and script I used test_commit.tar.gz:

nextflow_21.10.6 ~/p/scil/s/noddi_flow/main_commit.nf --input ../i_noddi_flow/ -with-report report.html --nb_threads 8

Then I check the report.html

If you change nb_threads (4 or 8) the report is very confusing. check reports.tar.gz

I tried also to run directly the python script with this command line:

scil_tractogram_commit.py tracking.trk dwi.nii.gz bval bvec     results_bzs/ --tol 30 --nbr_dir 500     --nbr_iter 500 --in_peaks peaks.nii.gz --in_tracking_mask brain_mask.nii.gz     --perp_diff 1.19E-3 0.85E-3 0.51E-3 0.17E-3     --iso_diff 1.7E-3 3.0E-3     --processes 8 --keep_whole_tractogram

Still quite understand why It does not use all the processes I allow him to use.

Thank you in advance for your help Arnaud

[arnaudbore]

@fullbat @daducci Any ideas ?

[daducci]

Hi Arnaud,

We are trying to understand the issue. I have some questions for you.

• To properly interpret your reports, shall we read the values reported as "Run times" or "CPU-Hours"? They seem to go in opposite directions and I don't have experience with Nextflow reports. Sorry!
• To understand where a possible bottleneck is, would it be possible to split the time profiling into two parts? Specifically, we would like to know the execution time (with different threads of course) only when running the function "tck2dictionary.run()" and when running the fit. That would help us a lot.

[arnaudbore]

Hi @daducci @fullbat @nightwnvol ,

Sorry for the delay. I've been testing a lot of things without being able to totally split the time profiling. I've tried something else. I run the script still using nextflow but I run it on beluga which is one the servers available on compute canada. Their reports helped me understand better.

I run the flow three times.

1. I set the number of threads to 8 for both trk2dictionnary and fit

2. Then I set the number of threads to 1 for trk2dictionnary and then 8 for the fit.

3. Then I set the number of threads to 8 for trk2dictionnary and then 1 for the fit.

As you can see there is something off about the capacity of the fit to be multi-threaded. Maybe there is something in my code that is not set properly.

I also updated our requirements to fit your latest releases.

[daducci]

Which OS is installed?

[arnaudbore]

On my laptop it runs on an ubuntu 22.04 and on beluga centos 8.9.

[daducci]

So, do you get the same issue (i.e., fit always performed with only 1 thread regardless of the number specified in the settings) both on your laptop and a cluster?

[arnaudbore]

Ok, it seems to work on my computer I can clearly see these two processes being parallelized. I don't think I've got the correct design and the correct way to profile this. I'm closing this issue. Thank you for your help.