How to learn the pose regression Model?

[ryukyungjoon]

I want to get the pose regression model. So, I'll construct to the dataset. What dataset did you use to make your pose regression model? DeepFluoro Dataset? What are the 1 million composite images you used to create the pose regression model?

I would appreciate it if you could answer the part that can be answered.

Thank you.

[eigenvivek]

HI @ryukyungjoon, to train on the DeepFluoro dataset, see here: https://github.com/eigenvivek/DiffPose/blob/main/experiments/deepfluoro/train.py

Make sure you've installed diffdrr==0.3.9.

[ryukyungjoon]

Thank you for your quick response. After the training is completed, should the model be evaluated as evaluate.py or register.py ? What is the difference between the results of the two codes? When I looked at the code, it seems that both codes use Deepfluoro, the dataset that we used in the pose regression model. In order to evaluate it as a custom dataset, I would appreciate it if you could tell me as much as possible how the custom dataset should be constructed.

[eigenvivek]

register.py! This runs the test-time optimization in the paper. evaluate.py runs only the network on the DeepFluoro dataset.

Setting this up for your own custom dataset is currently not easy :) You would have to write a torch dataloader for your dataset, then rewrite train.py and register.py to work with your dataset.

Creating a simple interface for training your own DiffPose model from scratch is on the todo list for version 2! ETA on that is ~1 month.

[ryukyungjoon]

Right. As far as I understood in your paper, we also confirmed that neurovascular testing was successful with a pelvic learning model. If it is difficult to learn with a custom data set, I would like to create a pervic model with the pervic learning code you implemented, and perform evaluation and registration with the custom data set.

Do I have to correct it to train.py and relearn it even if I perform evaluation and registration? I wonder if it is possible to use the method of giving the parameters required for test as input in the custom dataset.

[eigenvivek]

No that's not quite right. A new model was trained for the neurovascular dataset (that code is here: https://github.com/eigenvivek/DiffPose/tree/main/experiments/ljubljana). Basically, for a new anatomical structure, you have to train a new model.

[ryukyungjoon]

That's right. I got it wrong. When I checked the answer and looked at the contents of the paper again, it was that the model made by changing only the dataset without changing the hyperparameters of the model showed good performance.

If so, I understand that it is common to use datasets that have not been used for training when evaluating the learned model, but did the evaluation proceed with the deepfluoro used for training?

[eigenvivek]

The setup for DiffPose is like this:

1. You get a preoperative CT from a patient (i.e., a 3D volume). You generate synthetic X-rays from that patient and train a model with the synthetic data.
2. You evaluate the model using real X-rays from the same patient.

You can totally try using a model that was trained on one patient and evaluating it on X-rays from another. In fact, I think that's an interesting future application of DiffPose. However, while the model need not be trained and evaluated on the same patient, it should be trained and evaluated on the same anatomy.

[eigenvivek]

Closing for now, please feel free to reopen if you have more questions