search

DeepRegNet / DeepReg

Medical image registration using deep learning
Apache License 2.0
552 stars 76 forks source link

Add tracker information for registration #510

Closed NMontanaBrown closed 3 years ago

NMontanaBrown commented 3 years ago

Subject of the feature

What are you trying to do and how would you want to do it differently? From #424 raised by @agrimwood

"Adding information from tracker as inputs to the network to constrain reg, or as different inputs. So a module to do this/process txt files..?"

Is it something you currently you cannot do? No, we do not have this.

Is this related to an issue/problem? No.

Has the feature been requested before? If yes, please provide a link to the issue.

No.

If the feature request is approved, would you be willing to submit a PR?

M

mathpluscode commented 3 years ago

@agrimwood could you describe this feature a bit further? What does the tracker mean here ;)

YipengHu commented 3 years ago

@agrimwood i can't recall this exactly either........ did we discussed tracked points?

agrimwood commented 3 years ago

It's not a clearly thought out suggestion to be honest. I had two specific ideas in mind, but am not sure they're worth implementing:

1) 2D-2D registration: Imagine you are acquiring a freehand ultrasound image sequence where the ultrasound probe is being tracked (e.g. passive optical tracking). The tracking data provides a coordinate system to each image, which may be helpful for registering one or more frames to a reference frame. In conventional registration, a global registration is often performed using the image coordinates prior to applying a registration algorithm to reduce errors. 2) 2D-3D registration: Imagine you are acquiring tracked freehand ultrasound as above. This time you perform multi-modality registration of the 2D ultrasound images to a static 3D volume (e.g. CT). Assume the tracked ultrasound coordinates give you the same spatial reference as a CT volume. Use the US tracking data to perform a global registration of 2D US to CT.

A DeepReg user could obviously perform the tasks above themselves. If DeepReg gets to the stage where it can perform 2D to 3D registration, or where it can 3D register two 2D images acquired in different imaging planes, then maybe there's a case for adding this feature...

mathpluscode commented 3 years ago

@agrimwood I feel like we are talking about extending from 3D-3D to 2/3D to 2/3D registration? Technically it should be just a matter of data loader and network. It could be a very important feature.

If that's the idea, we can close this and open a new ticket for 2D-2D registration first. What do you think?

agrimwood commented 3 years ago

@mathpluscode I agree - either 2D-2D, or 2D-3D (if that isn't already being implemented).

@YipengHu @mathpluscode I will open a new ticket unless you have any objections