Closed dorkylever closed 4 years ago
Hi Kyle,
It's been a while since I last ran embryo pipelines, so I can't guarantee that the following instructions still work with the current code, but it should serve as a good guideline. I found that because of the potential variability between embryos, the non-linear stages could not be performed simply with a "build-model" approach. I.e., the approach where all embryos are registered towards the previous average (and repeated a couple of times to keep refining the target average). This didn't work, because the average model after the LSQ12 stage (affine registration) was not sharp enough, and non-linear registration could not escape the ambiguity in the blurry areas. So instead I first ran a non-linear pairwise stage. Here all samples are non-linearly registered towards all other samples and the average transform is applied to this sample. This results in a much better average, and can now be used for the regular "build-model" approach. To use this, specify the following flag for MBM.py:
--registration-method ANTS --registration-strategy pairwise_and_build_model
The content of the protocol file I used is as follows:
"gradient";False,True;False,True
"similarity_metric";"CC,CC";"CC,CC"
"weight";"1,1";"1,1"
"radius_or_histo";"3,3";"3,3"
"transformation";"SyN[0.2]";"SyN[0.2]"
"regularization";"Gauss[2,1]";"Gauss[2,1]"
"iterations";"200x200x200x100";"200x200x200x150"
"useMask";True;True
Hope that helps! Matthijs
Hi Matthijs,
Thank you for your help!!
The pairwise_and_build_model command was really helpful. I tried your full command with our uCT scans but I ran into an error during ANTs. Myself and Henrik Westernberg were able to use minctracc instead of ANTs for nlin registration and I came up with a merge between your and Henrik's command (which I put in the wiki for other people trying to use Pydpiper 2.013 with uCT scans of embryos)
Kind Regards, Kyle Drover
Hi Matthijs,
Just as a reference, when I run the following MBM.py run with ANTs:
MBM.py --pipeline-name TCP_ANTS --mem 100 --num-executors 40 \ --init-model /home/kyle_arkell/mouse_atlas/image.mnc \ --registration-method ANTS --registration-strategy pairwise_and_build_model \ --nlin-protocol /home/kyle_arkell/embryo_protocol.csv \ --no-run-maget --maget-no-mask --no-common-space-registration --verbose --config-file /home/kyle_arkell/arkell.cfg \ --lsq12-protocol /home/kyle_arkell/Henrik/TCP-1700003F12Rik/MBM_lsq12_embryo_protocol.csv \ --output-dir /home/kyle_arkell/TCP_fix_ANTS/ --files /home/kyle_arkell/TCP_short/*mnc
It fails at ~Stage 650 because it cannot reach oscillation (presumably because its a bad transform):
ERROR in Stage 650: ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc Logfile for (potentially) more information: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/../log/ANTS/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.log
cat /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/../log/ANTS/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.log
Stage 650 running on compute-5-2.local (PYRO:obj_76bb58a5b89c484299445e678d2004b4@10.1.255.242:45690) at 2019-12-03 13:11:52.935470: ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc Run Reg values 2 Fixed image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc Moving image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc Metric 0: Not a Point-set Fixed image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc Moving image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc similarity metric weight: 1 Radius: [3, 3, 3] radius: [3, 3, 3] Fixed image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc Moving image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc Metric 1: Not a Point-set Fixed image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc Moving image file: /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc similarity metric weight: 1 Radius: [3, 3, 3] radius: [3, 3, 3] Use identity affine transform as initial affine para. aff_init.IsNull()==1 Use identity affine transform as initial fixed affine para. fixed_aff_init.IsNull()==1 Continue affine registration from the input affine_opt.use_rotation_header = 0 affine_opt.ignore_void_orgin = 0 transform_initial: IsNotNull():0 OptAffine: metric_type=AffineWithMutualInformation MI_bins=32 MI_samples=32000 number_of_seeds=0 time_seed=1575339129 number_of_levels=1 number_of_iteration_list=[0] graident_scales=[1,1,1,1,1,1,1,1,1,1,0.0001,0.0001,0.0001] is_rigid = 0 mask null: 0 maximum_step_length=0.1 relaxation_factor=0.5 minimum_step_length=0.0001 translation_scales=0.0001 opt.transform_initial.IsNull(): 1 opt.use_rotation_header: 0 opt.ignore_void_orgin: 0 input affine center: [-1.48712, -6.4531, -6.72982] input affine para: [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0.9454505464815801, 3.066544297337104, 2.0875171507811885] level 0, iter 0, size: fix[347, 405, 602]-mov[347, 405, 602], affine para: [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0.9454505464815801, 3.066544297337104, 2.0875171507811885] does not reach oscillation, current step: 0.1>0.0001 final [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0.9454505464815801, 3.066544297337104, 2.0875171507811885] outputput affine center: [-1.48712, -6.4531, -6.72982] output affine para: [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0] initial measure value (MMI): rval = -0.0136669 final measure value (MMI): rval = -0.000491349 finish affine registeration... Requested Transformation Model: SyN : Using SyN diffeomorphic model for transformation. Grad Step 0.2 total-smoothing 1 gradient-smoothing 2 setting N-TimeSteps = 1 trunc 256 ScaleFactor 8 nlev 4 curl 0 allocated def field 1 0 0 0 1 0 0 0 1
Its at this level 200
It doesn't look like that's why it's failing (it's not actually running any affine iterations!). Most likely things are being killed by your scheduler for using too much memory as the nonlinear registration begins ... are you getting message about executors being killed? If so, best to check the executor logfiles for clues ...
Hi Ben,
I tripled the resources given to the scheduler. i.e. running the following shell script locally:
cat launch_MBM_fix_ANTs.sh
MBM.py --pipeline-name TCP_ANTS --mem 400 --num-executors 20 \ --init-model /home/kyle_arkell/mouse_atlas/image.mnc \ --registration-method ANTS --registration-strategy pairwise_and_build_model \ --nlin-protocol /home/kyle_arkell/embryo_protocol.csv \ --no-run-maget --maget-no-mask --no-common-space-registration --verbose --config-file /home/kyle_arkell/arkell.cfg \ --lsq12-protocol /home/kyle_arkell/Henrik/TCP-1700003F12Rik/MBM_lsq12_embryo_protocol.csv \ --output-dir /home/kyle_arkell/TCP_fix_ANTS/ --files /home/kyle_arkell/TCP_short/*mnc
And it failed at the same step as above.
Do you think I shuld increase my h_vmem more? It seems weird as test_MBM_and_MAGeT.py only needed 15gb of h_vmem shen I ran it.
Did you check the executor logs to see if this indeed the problem? It's possible that, for instance, some other memvars need to be set, e.g. mem
, not just vmem, but you'd want the error messages to figure out how your grid's scheduler is configured. You could also simplify things by running an ANTS command by hand, and possibly under top for memory profiling, e.g., top -v ANTS ...
Hi Ben,
I'm running the commands locally on a subnode of our cluster (arkell.cfg is just basically an empty config file). Running a single ANTs command alone with the same config as above i.e:
Will result finish with no error being raised.
Is the executor log you're talking about?
check_pipeline_status.py TCP_ANTS_uri Total number of stages in the pipeline: 1557 Number of stages already processed: 685
Number of active clients: 20 Number of clients waiting in the queue: 0
Currently running stages (20): 644 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
646 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e9_wt/resampled/kat233_e9_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e9_wt/tmp/kat233_e9_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat233_e9_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
648 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e1_wt/resampled/kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e1_wt/tmp/kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
650 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
652 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e7_wt/resampled/kat243_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e7_wt/tmp/kat243_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e7_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
685 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/tmp/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/transforms/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
654 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e1_wt/resampled/kat288_e1_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e1_wt/tmp/kat288_e1_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e1_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
687 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/tmp/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/transforms/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
664 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e7_wt/resampled/kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e7_wt/tmp/kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
689 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e1_wt/resampled/kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/tmp/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e1_wt/tmp/kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/transforms/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e1_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
658 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e3_wt/resampled/kat288_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e3_wt/tmp/kat288_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e3_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
690 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/resampled/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/tmp/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat243_e4_wt/tmp/kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e6_wt/transforms/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat243_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e6_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
660 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e4_wt/resampled/kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e4_wt/tmp/kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
725 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e2_wt/resampled/kat288_e2_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e2_wt/tmp/kat288_e2_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/transforms/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e2_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
662 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e6_wt/resampled/kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e6_wt/tmp/kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
727 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e4_wt/resampled/kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e4_wt/tmp/kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/transforms/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e4_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
728 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e6_wt/resampled/kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e6_wt/tmp/kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/transforms/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e6_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
729 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e7_wt/resampled/kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e7_wt/tmp/kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/transforms/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e7_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
666 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e9_wt/resampled/kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/tmp/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e9_wt/tmp/kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e3_wt/transforms/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e3_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
730 ANTS 3 --number-of-affine-iterations 0 -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e9_wt/resampled/kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled.mnc,1.0,3]' -m 'CC[/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/tmp/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,/home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat288_e9_wt/tmp/kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled_fwhm0.027_dxyz.mnc,1.0,3]' -t SyN[0.2] -r Gauss[2,1] -i 200x200x200x150 -o /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_processed/kat233_e7_wt/transforms/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_ANTS_to_kat288_e9_wt_N_I_lsq6_avg_lsq12-resampled.xfm -x /home/kyle_arkell/TCP_fix_ANTS/TCP_ANTS_init_model/image_mask/resampled/kat233_e7_wt_N_I_lsq6_avg_lsq12-resampled_mask.mnc
Number of runnable stages: 160
Number of failed stages: 0 Number of failed/lost/dead executors: 0
Memory requirement of runnable stages: [17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347, 17.9434347] Memory available in registered clients: [17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434, 17.9434]
This is top summary during ANTs when I call MBM.py:
top - 12:43:13 up 298 days, 12:25, 2 users, load average: 11.34, 12.07, 12.11 Tasks: 315 total, 6 running, 308 sleeping, 0 stopped, 1 zombie Cpu(s): 99.9%us, 0.1%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 49558168k total, 35489860k used, 14068308k free, 10600k buffers Swap: 1023996k total, 703828k used, 320168k free, 287840k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24959 kyle_ark 20 0 5774m 4.6g 1536 R 800.0 9.8 417:25.36 ANTS 16210 kyle_ark 20 0 4136m 3.1g 17m R 100.0 6.6 81:01.19 ANTS 14006 kyle_ark 20 0 5612m 4.5g 1532 R 97.4 9.6 243:15.08 ANTS 25628 kyle_ark 20 0 17.1g 16g 1384 R 97.4 34.0 760:34.27 ANTS 9379 kyle_ark 20 0 5613m 4.5g 1532 R 89.7 9.6 140:23.12 ANTS 14960 kyle_ark 20 0 7915m 75m 1864 S 2.6 0.2 45:50.05 MBM.py
Hi Everyone,
Pydpiper has been used for analysing micro-CT (uCT) scans of embryos (Wong et al 2014 -https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4050698/#DEV107722C34), however, no registration protocols (i.e. csv files) or example runs are documented.
My supervisor (Ruth Arkell) and I will discuss these settings with representatives of the first author and I will update the documentation myself if given permission.
In the meantime, does anybody have protocols that that are related to scans of embryos?
Kind Regards, Kyle Drover (PhD Candiate within the Arkell Group at the Australian National University)