search

ins-amu / scripts

prepare data for TVB
MIT License
21 stars 12 forks source link

Parallelize tckgen across nodes, combine with tckedit #47

Closed maedoc closed 8 years ago

maedoc commented 8 years ago

tckgen runs parallel tracking within a single computer, but it could be parallelized across nodes as well by running tckgen N times, each with different seed & output file, then combining files like

$ tckedit whole_brain_part*.tck whole_brain.tck

This seems to be an I/O bound step, reading all input files and writing back to output file.

timpx commented 8 years ago

the use case I see for the cluster would more be one subject by node, and then in a node using different cpus. Would it be so useful to parallellize tckgen across nodes ?

maedoc commented 8 years ago

No, this was partially a theoretical question, in the case where many nodes are free and we want to run through a patient as quickly as possible.

A single node is probably a better use compromise between run time & resources.