search

saiboxx / chexray-diffusion

Code for "Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis" @ PAKDD 2023
MIT License
46 stars 8 forks source link

Request for how to use the Chest X-ray report-conditioned semantic diffusion model #10

Open chuhuan88 opened 5 months ago

chuhuan88 commented 5 months ago

It's an honor to see such a wonderful project of yours,I can't wait to use the model you shared.But when I was using the model cheff_diff_t2.pt, I ran into a problem.

When I used this model to do a text-guided repair of the mask area, I found that the generated image was fuzzy and the mske area failed to be repaired.

Can you tell me how to use this model correctly?

Thank you for such excellent work,and looking forward to your reply.

saiboxx commented 1 week ago

Hi @chuhuan88,

Thanks for the interest in my work. I apologize for the late reply. I was on a longer vacation and additionally left academia, so I check GitHub only sporadically these days around.

In our published paper, we have not evaluated the quality of text-guided masking. All in all, our text-guided model is more of a prototype and the most likely case is that it just fails to perform the desired task.

There is currently a lot of research going on when it comes to CXR report generation and image manipulation/modeling. I would in all honesty investigate those, more recent, methods, if you are interested in T2I. Our work is effectively from 2022, which in accelerated times like these is an eon, and you will definitely benefit from improvements made since then.

Cheers, Tobias