difference with DiffDRR?

[anxingle]

Dear eigenvivek: Thank you very much for your excellent paper! I just wonder how about just use DiffDrr withGeodesic pose regression losses + Multiscale NCC ? I read from your tutorial .

Since I work in a surgical robotics company, I find train a network for every CT scan is really time-consuming. (You can image if every spine surgery required training a CNN to continue registration and operate... My boss will kill me). So I perfer using DiffDRR in my work.

4.5 Implementation Details: For each CT scan, a patient-specific ε was trained from scratch using synthetic X-ray images rendered on the fly. It seems DiffDRR is more practical...

[eigenvivek]

Hi @anxingle thank you for the kind words. You're right - you can just use DiffDRR to perform your registration. The point of training the the CNN in DiffPose is to get ultra-accurate initial pose estimates. The actual registration (called "test-time optimization" in the CVPR paper) is done using DiffPose.

If you have a good guess for the initial pose, you can just run DiffDRR using the tutorial notebook you liked above.

Note that you can't use the geodesic pose regression losses with DiffDRR - those are predicated on knowing the ground truth pose, something you only have during training on synthetic images. So to do registration with DiffDRR, I use an image similarity metric (other metrics beyond multiscale NCC are implemented in the DiffDRR package).

[anxingle]

Got it! Thank you so much . The advent of your work announces that registration entered the era of Differentiable . It will be a foundational work just like AlexNet.