search

SuperElastix / SimpleElastix

Multi-lingual medical image registration library
http://simpleelastix.github.io
Apache License 2.0
513 stars 149 forks source link

Computer locked when running groupwise registration #118

Closed thatguy14 closed 5 years ago

thatguy14 commented 7 years ago

Hello,

I am having an issue running the groupwise registration. I am trying to register a large image set (29 3d volume with 16,515,072 voxels). I load these image sets and follow the example for groupwise registration by making them a 4d vector. I also have a fixed mask that limits the number of included voxels to about 1 million. I ran this code end of day Friday and my computer locked up with the occasional unfreeze but basically unusable. I decided to just let it run. Today I came in and tried to do anything on my computer but it was completely locked - couldn't ctrl+alt+del or move the mouse so had to do a hard reset by holding down the power button.

I doubt this is related to SimpleElastix and more related to using large data sets but I wanted to make sure by asking here first.

PertuyF commented 7 years ago

Hi @thatguy14 , What you describe sounds like you filled your RAM and your system is swapping. Can you monitor RAM usage?

thatguy14 commented 7 years ago

I ran it again yesterday and yea my ram was maxed out. Is there a solution that I could try (other than just getting more ram)? Is it I just have too many images?

PertuyF commented 7 years ago

Well, there are several points to check here that could allow you to use this method:

Maximize free RAM

Tweak Elastix

If it is not enough, there are some things you can change in your registration.

You do not necessarily need to do all at once. I recommend that you first reduce drastically the number images (lets say 10), then monitor the amount of memory used by Elastix during registration. If you calculate a per-image ratio of used memory, it should scale with the number of images if you use the exact same parameters. This will give you an idea of the maximum number of images you can use with this approach.

Change your approach

If you're still maxing out your RAM with only 10 images, or if it is mandatory for you to use all 29 image for your average image, then come back here. Other approaches (a bit more complex) can be used to achieve a similar results.

thatguy14 commented 7 years ago

Thanks for the help!

I will try your suggestions of using less images. I may also try cropping the images to the region of interest as this will reduce the size of each image by about 5x. I will look into the other Elastix Parameters.

Thanks again

PertuyF commented 7 years ago

You're welcome :) And let me know how it goes.

If needed there are methods that only require to register images by pair. It is less elegant than this single groupwise registration, but it works well.

thatguy14 commented 7 years ago

Hi again!

I ran the same script I used previously but doing as you suggested using less images. In this case I used half of my data set (14 images) and an image mask. It ran to completion successfully though I seem to forgot to include the write to file command so I don't have the results to look at. Oh well, at least it works.

To give you a sense of my data (not familiar with what you know so if you need any clarifications I'd be happy to supply them) I am looking at dynamic contrast enhanced (dce) MRI data. Originally I was doing pair-wise registrations to some fixed image in the set. This has been working fairly well but I always like exploring new avenues. A particular problem I have is that the dynamic contrast enhanced data includes a set of images that are acquired before hand that act as a sort of correction factor (they are called T1 maps - again not familiar with what you know so I wanted to keep it simple). Ideally, the 29 dce-mri data would be co-registered to the 5 correction images. I figured this might be a good application for groupwise registration since they all need to be co-registered to each other for high quality data.

Sorry for the long winded explanation but I figured it might help if I gave some context to what I am trying to accomplish. There can be variations in the direction of movement from one image to the next so I think it might be worthwhile to try and do the groupwise using ALL the images... What do you suggest? Also, sort of unrelated, is it possible in SimpleITK and SImpleElastix to implement custom optimization metrics? I know you can do it using ITK but I am not skilled enough of a programmer to look into that at this point in time.

PertuyF commented 7 years ago

Thanks for the context :) I have a background in Biology and Medical Imaging, but I am not an expert of MRI.

Groupwise registration

I am not sure groupwise registration is what you need here. It is meant to register all images at once as a population, to each image at once (as references). This leads to an average deformation where all images will look alike. But where the deformation will be specific for a given population, and could not be extended to additional images. It is indicated when no image in the population could serve as reference. For example a series of individuals where each has specific variations that should not be given a particular weight in the final registration.

In your case you have a reference (or 5? that point I do not understand completely, if you have time for an explanation). So I think the best approach would be to register your 29 dce-mri images to that reference. More practically, as you cannot register all images at once, you will likely not be able to use groupwise registration for your problem anyway.

The question here is more whether you should use one T1 map as the reference? or all 5? or an average of the 5? (which may be produced using groupwise registration) but I need more explanation on that part.

Custom metric

No you can't use a custom metric with ElastiX, but there are different metrics available in ElastiX. The more general metrics are described in the manual, but all are explained in the documentation (Careful, some have very specific use case).

What you can do is to use MultiMetricMultiResolutionRegistration that allow to compose with several metrics at the same time, give them weight... It can make the difference sometimes, that's useful! (and time consuming...)

thatguy14 commented 7 years ago

I'll try and explain the process better.

Dynamic contrast enhanced MRI This technique is used to characterize the vascular function of tumours (or atleast that is one application). You acquire an image (called the pre-contrast), inject a contrast agent that accumulates in the tumour (due to leaky biology of tumour vasculature) and causes a intensity increase (called enhancement). You then acquire a time series of post-contrast images. You can then look at a specific voxel and plot the intensity vs time and fit a model. However, movement can cause fluctuations in the signal values since you may not be looking at the same voxel from one time point to the next. That's where registration comes in and I have had good success so far correcting my data using rather standard registration algorithms. I thought groupwise registration would be a good application since these images are all very similar except for movement and the local intensity changes due to the contrast.

T1 Mapping Images I think I may have confused things here for you. I have a series of 5 images that have varying contrast-to-noise ratios. The idea here is to fit a signal equation to each voxel to get a value called the T1 of the tissue and to use that value in the fitting process mentioned above. I was not sure if it would be possible to combine the set of DCE images with the T1 images since they do have different intensity profiles but I thought I would try.

Reference Image This is in reference to your question. I don't have a specific reference. There are typical things people do for registering DCE data to itself and registering T1 data to itself but I have almost no papers that refer to registering the DCE set to the T1 set. It shouldn't be that much of an issue and I have had some success in this endeavour. However, again, I just wanted to see if this might be a good application for groupwise registration. It may be that I don't entirely understand the groupwise algorithm. I cannot use the T1 map as the reference as it is a generated image through a fitting procedure - I don't think it would work well but have no tried it and have not seen it done.

Appreciate the time, I will look into the metrics! MI has been working well so far and seems rather robust. Any other clarifications you have or comments would be happy to hear, especially about using the groupwise registration (even if it is just for the DCE data set). Specifically, do you have any experience with how it performs when the images to be registered have different contrasts?

PertuyF commented 7 years ago

Thanks ! that's a bit clearer now.

Metrics

You're right, MI is a good choice in this case, as it is well suited not only for mono-modal but also multi-modal registration. The ElastiX manual also mention that NormalizedMutualInformation (NMI) might be interesting too.

The NMI measure is, just like MI, suitable for mono- and multi-modality registration. Studholme et al. [1999] seems to indicate better performance than MI in some cases.

I would not be surprised that DCE/T1 registration works. As long as the information is there (most of the structures are present on both images), the influence of the contrast itself will not be that important. I often faced unjustified a priori on this point based on our human perception of what "similar" images looks like.

Groupwise registration

I won't explain the algorithm used by elastix (not sure I can be clear enough), but an approach that produce a similar result an is more intuitive to me. Basically, you start with a population of images representing variations of the same structure (along time or across individual for example). Here is the process:

  1. take one of these as moving image
  2. register it pairwise to all images from this population (including the moving image itself)
  3. compute the average of all pairwise transforms
  4. apply the mean transform Repeat it with all images and you'll have a similar result to that of elastix groupwise registration (but it does all in one step using VarianceOverLastDimensionMetric).

The point here is to minimize structure variations across the population, that allow either to compare images easily (as each voxels corresponding to the same structure should now have the same index in each images). Or to produce a voxel-wise average image of the population, that will improve the signal/noise ratio (a lot!) while keeping the details of shared structures.

Back to your case

Whether this kind of approach suits your needs or not is up to you, but depending on the exact nature of your images, I may have a few recommendation.

Hope it may help a bit.

thatguy14 commented 7 years ago

Very thorough and appreciate the suggestions. This is a very large world for a grad student like me where my project isn't really about registration but it has become an integral part of the exploration process!

I will look into some of these options. I have currently been using 3DSlicer but would love to see if I can get better results faster using some other method. Appreciate the help!

Oh actually one last thing - I think I know the paper that the TransformRigidityPenalty is based on (I think it's Rohlfing et al 2003) but do you know where to find more info about it?

kaspermarstal commented 7 years ago

The paper describing the elastix implementation is M. Staring, S. Klein and J.P.W. Pluim, "A Rigidity Penalty Term for Nonrigid Registration," Medical Physics, vol. 34, no. 11, pp. 4098 - 4108, November 2007. You can read it here :)