search

wustl-cig / DOLCE

DOLCE, ICCV2023. Pytorch Implementation.
MIT License
36 stars 6 forks source link

How to ensure 3D consistency (inter-slice consistency) #9

Closed Zhentao-Liu closed 2 months ago

Zhentao-Liu commented 3 months ago

You use diffusion model to generate 3D CT volume slice by slice. You use parallel beam forward projection to guide the diffusion inference sampling process, the same as DiffusionMBIR [1], DDS [2], but how do you ensure 3D consistency? or inter-slice consistency. Your results demonstrate you are inter-slice consistent. That's wired. I am really curious.

[1] DiffusionMBIR: H. Chung, D. Ryu, M. T. McCann, M. L. Klasky, and J. C. Ye, “Solving 3d inverse problems using pre-trained 2d diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22542–22551, 2023 [2] DDS: H. Chung, S. Lee, and J. C. Ye, “Decomposed diffusion sampler for accelerating large-scale inverse problems,” arXiv preprint arXiv:2303.05754, 2024

Zhentao-Liu commented 3 months ago

Maybe you should use cone beam forward projection ? BUT I think it is still very difficult to ensure 3D consistency.

JiamingLiu-Jeremy commented 2 months ago

Hi @Zhentao-Liu,

Thank you for your interest in our work. We observed structural consistency when using pretrained DPMs conditioned on FBP or RLS, especially when tested on luggage phantoms. But as you pointed out, our work is fully compatible with [1] or [2], where one could add extra slice-wise constraints to improve structural coherence.