Xrayto3DShape: A Benchmark for Biplanar X-rays to 3D Shape Reconstruction
Paper · Benchmark . Code Example

News and Updates

• [02/09/2024] We have released the trained models. [Trained Models]

• [02/09/2024] Benchmark evaluation logs and results of Xrayto3D benchmark has been updated! [Results] [wandb]

• [12/20/2023] The Xrayto3D benchmark paper has been accepted at NeurIPS 2023 Dataset and Benchmark Track! [Openreview]

Xrayto3D-benchmark

This is a pytorch-based Python package for Biplanar X-ray to 3D Shape Reconstruction. We aim to become a comprehensive benchmarking tool for developing and evaluating methods and architectures specific to this problem.

Currently, we provide tools to train and evaluate on 4 different bone anatomies, using publicly available CT-segmentation datasets. We also define tasks for domain shifts to evaluate the robustness of these methods.

Various Encoder-Decoder Architectures on volumetric grid-based representation are reimplemented and benchmarked.

Getting Started

Prerequisites

To install the required packages

conda create env --name xrayto3dbenchmark-cuda_11 --f  benchmark-requirements-cuda_11.yaml
# or
conda create env --name xrayto3dbenchmark-cuda_10 --f benchmark-requirements-cuda_10.yaml
conda activate ...

Install the XrayTo3DShape package

pip install XrayTo3DShape  

Sample Training script call

python train.py  configs/paths/femur/30k/TotalSegmentor-femur-left-DRR-30k_train+val.csv configs/paths/femur/30k/TotalSegmentor-femur-left-DRR-30k_test.csv --gpu 0 --tags model-compare --size 128 --batch_size 4 --accelerator gpu --res 1.0 --model_name MultiScale2DPermuteConcat --epochs -1 --loss DiceLoss  --lr 0.002 --steps 4000 --dropout

Prepare Datasets

A separate repo for preprocessing data has been created here. The instructions for downloading and processing datasets are described here.

Usage

Examples to help you get familiar with the XrayTo3DShape package for quick use, evaluate an existing architecture on your own dataset, or benchmark new architectures.

Quick Start

• Beginning Examples
  • Sample Model Prediction
• Customize Datasets
• Regarding Orientation alignment between Input X-rays and Output Segmentation

Training

See bash_scripts dir

Evaluation

• Domain Shift
• Angle Perturbation

  Benchmark Results

  wandb results page

Model Zoo

TODO: Add pre-trained models

Acknowledgements

We took considerable inspiration and references from:

• USB
• MONAI

Various adaptation of code(or code snippets) from these sources:

• FracReconNet
• VerSe
• torch-template

Various sections of the code uses following external codebases:

• SpatialConfiguration-Net
• surface-distance
• TotalSegmentator