search

eigenvivek / DiffDRR

Auto-differentiable digitally reconstructed radiographs in PyTorch
https://vivekg.dev/DiffDRR
MIT License
148 stars 20 forks source link

Question: Non-centered copped X-Ray #285

Closed poretga99 closed 1 month ago

poretga99 commented 4 months ago

I want to perform DRR rendering with reference geometry given in transformation matrices instead of the C-arm parameters and can't get the same result as our internal rendering engine.

Given the following matrices given in DICOM coordinate system:

TPos: 4x4 homogenous matrix, representing the detector position in WCS SPos: 3x1 vector, representing X-Ray source position in WCS

How can one properly construct the geometry? I can expose the SSD by calculating the distance from source location (SPos) to X-Ray detector plane, but the ray does not need to directly intersect with the detector midpoint.

eigenvivek commented 4 months ago

Hi @poretga99, it's always a fun time transforming from one person's affine geometry to another's :) DiffDRR is able to support all the geometry geometry you have (including the principal offset point), but it'll be annoying to get it back in term of a single set of intrinsic/extrinsic parameters. In case all reverse engineering fails, just note that the internal rendering system allows you to directly pass the 3D world coordinates of the X-ray source and detector pixels (source and target):

https://github.com/eigenvivek/DiffDRR/blob/9bda058241aa084f83dcc73c2eeb2927f1715481/diffdrr/drr.py#L131-L138

I'm also happy to help reverse engineer the intrinsic/extrinsic parameters if you want! Just let me know which route you'd prefer to go.

poretga99 commented 4 months ago

Before deciding on which route to choose, maybe I should describe the setup first.

I have a 2D3D registration problem. Given two reference XRays and CT image, I have to find the optimal position of CT in space.

As far as I’ve gone through the code and documentation, you use RAS convention, whereas we use DICOM, which will be another problem by itself.

Each of my XRays has the following information:

  1. TPos: 4x4 matrix that maps the left lower corner of the XRay into the WCS (rotations included. XRay is assumed to be in the XY plane)
  2. SPos: 3x1 vector representing the XRay source position in WCS
  3. Detector geomery: size and pixel spacing

3D CT scan has the following information:

  1. Origin: 3x1 vector describing the location of the left most voxel (DICOM standard, ITK uses such convention)
  2. Spacing
  3. Size

In my case, XRay SPos (source position) may not be coincident with the detector midpoint. This typically happens when you crop the XRay image to the desired region of interest.

What would your suggestion be on how to proceed? I can also provide the images and geometry as a result of our current rendering engine.

poretga99 commented 4 months ago

I managed to reconstruct the desired DRR by directly passing the detector points and source position as per your suggestion. Thanks for the help!

Even though, I would be interested in the intrinsic/extrinsic parameter extraction.

eigenvivek commented 1 month ago

Hi @poretga99 sorry for the slow reply on this. If you still want to find the intrinsic/extrinsic parameters, feel free to send me an email and we can try to work through it.

I recently worked on another project where I knew the source/target points in 3D for every pixel in a set of X-ray images, but not their intrinsics/extrinsics. Trying to solve for them proved to be a fruitless blackhole with minimal benefit, so directly using the source/targets is what I've been doing for my own stuff.