search

pnlbwh / dMRIharmonization

Multi-site dMRI harmonization
Other
43 stars 14 forks source link

Resume stopped template creation #34

Closed fedemoro closed 4 years ago

fedemoro commented 4 years ago

I exceeded walltime limit during the template creation and my job was killed. I'm wondering whether it is possible to resume the process from where it was stopped.

 Starting ANTS registration on max 4 cpucores. Iteration: 4 of 4
 Progress can be viewed in job*_3_metriclog.txt
--------------------------------------------------------------------------------------

Many thanks

tashrifbillah commented 4 years ago

Hi @fedemoro , I don't know what walltime is. I know it sounds upsetting, but you may have to start all over again. However, your question is beyond our scope to support. You can try asking this on ANTs issues.

Nonetheless, how many subjects do you have and how long did it take so far? How many processors did you use?

fedemoro commented 4 years ago

Hi @tashrifbillah, I've the following: nproc 4 ref: 24 tar:17 Last time it took 5 days to finish the template creation. We have several limitations on the cluster and if I try to increase the number of processors I exceed the memory limit

tashrifbillah commented 4 years ago

I can give you a way around. Can you not use your workstation/personal computer to do the template creation and then move your template folder to the cluster? That way, you would have all control over the computational resources.

Another thing to note, for our template creation step, you need matching set of reference and target subjects. Your ref and tar have different number of subjects.

fedemoro commented 4 years ago

subjects. Your ref and tar have different number of subjects

thanks for clarifying this, I thought the matching was referring to age, gender IQ but that you can use different number of HCs between target and reference.

tashrifbillah commented 4 years ago

You first create a template with matching set of reference and target cases (same number of). Then, you apply the template to harmonize all of target cases.

If the target site consists of only the cases used in template creation, then template creation and harmonization can be done in one step with --create and --process flags. If you have more cases in target sites, then it becomes a two step process. Create template first and then harmonize.

What I said, is mentioned here and here.

Hope this makes more sense.

fedemoro commented 4 years ago

Thanks, I need to harmonize 6 sites (each one with 16 HCs) to our reference site that has a total of 80 HCs. So, for the template creation I'll need to select the best matching for each target site among the subjects of our reference site.

tashrifbillah commented 4 years ago

I hope by HC you mean subjects.

The answer is yes, just use different names for template folder i.e template1/, template2/ etc. That way, you won't have to make a copy of reference images.