Projection parameters

[pimatysiak]

Hi Robert, sorry to bother again but I am having doubts about my understanding of some of the parameters.

My rough understanding is that the extrinsic matrix, in general, gives the position/orientation of the camera capturing an image. That is why it would make sense in my head to have one such matrix for each projection.

In your dataset there seems to be a global extrinsic matrix to shift to camera space, and 3 different matrices to shift into volume space (whether pelvis or either femur). Could you please give a brief explanation as to how you obtained these matrices, what the global one represents, and where the shift takes us (is it the center of the volume? one of the corners?). Trying to generalise to some other data has proven a little difficult for me..

Thank you.

[rg2]

Hi @pimatysiak ,

You are correct that the extrinsic matrix describes the pose of a single camera's projective frame with respect to some other frame. Usually it makes sense for this matrix to be helpful in the context of the camera's setup, e.g. its pose with respect to other co-calibrated cameras or its pose at a certain time for a camera undergoing some known rigid motion, etc. The extrinsic matrix for all of the images in this dataset was estimated so that the rotation about the x-axis in the extrinsic frame corresponds to an orbital rotation of the C-arm. This is useful for an efficient intraoperative workflow, such as the one described in [1]. The process of estimating this extrinsic matrix is described in [2] with source code available in that repository.

Since the pelvis, left femur and right femurs are all separate rigid objects that may move non-coherently, there is a separate pose needed to map each of these objects from their CT coordinates into the extrinsic frame. These transformations are found through a process known as registration, which is the main application of the companion paper of this repository [3].

[1] https://arxiv.org/abs/1903.09339 [2] https://github.com/rg2/xreg/wiki/Example%3A-C-arm-orbital-rotation-axis [3] https://arxiv.org/abs/1911.07042

[pimatysiak]

Thank you so much for all the information! I'll be sure to read this very carefully.

[rg2]

You're welcome!

[pimatysiak]

Hi Robert, hope you're doing well.

I have followed the instructions you provided and it worked really well, however I still am not completely sure about obtaining the values for the global extrinsic matrix (the single one you have in the 'proj-params' subsection of your dataset, not the extrinsic matrices for each individual views).

I noticed when I use the function "xreg-proj-est-orbital-rot" to obtain and/or update the projection parameters for the various x-ray images I have available, there is a message diplaying "Updated extrinsic matrix:" [...]. Is this matrix supposed to be the global one? I am having doubts because it perfectly matches the extrinsic matrix for the first view in my series, and using that in the rest of my experiments leads to wrong projections results.

Note however that all the x-ray images I have available for my dataset were captured from an AP view with mostly translation varations, and hardly any rotations, which could potentially explain why the center point could be confused with one of the actual views.

If my explanation makes no sense I can provide additional values and visuals if you prefer.

Thank you

[rg2]

Hi @pimatysiak , it will be helpful to completely describe your dataset along with some links to the data or visualizations, and very specifically describe the problem you are trying to solve. Otherwise, it is challenging to provide sufficient guidance.

[pimatysiak]

Yes of course, thank you for the reply! The dataset I'm currently working with consists of a single CT volume (centred around the pelvis) and 20 x-ray images captured mostly from an AP viewpoint, with light translation of the C-arm. After manually annotating landmarks for both the volume and all x-rays, I go through the protocol you linked as [2] in your above message, and end up with pose estimations looking a little like this:

If I understand things correctly, the green points and line represent the camera position and orientation for each of the computed x-ray. My intuition is that these are close approximations to the "cam-to-pelvis-vol" matrices stored in the ground truth folders of each projection in your dataset. I'm not sure if that is correct either.

What I am after is instead the global extrinsic matrix you list in the "proj-params" section of your dataset. I imagine it corresponds to the yellow point computed and visible in the above image. While going through the xReg protocol in [2], the only time I see mention of an isolated extrinsic matrix is when running the function "xreg-proj-est-orbital-rot" on the file containing all pose predictions, and these appear when the verbose tag is set:

Am I correct in assuming this described extrinsic matrix is the one represented with the yellow dot? What confuses me is that, in the file all_pd.h5 containing all computed projections and respective parameters, the extrinsic matrix for the first projection perfectly matches the one displayed above:

Whereas none of the x-rays' projection dot seem to coincide with the yellow one.

I don't know if these details made it anymore clear to you, but I guess I am trying to replicate the structure of the dataset you provided, using my own. I find the discrepancy between your global extrinsic matrix and each projection's individual one to be much bigger than for what I obtain, and I cannot make sense of that.

Sorry if this is just even more confusing, and thank you in advance.