search

saiboxx / chexray-diffusion

Code for "Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis" @ PAKDD 2023
MIT License
44 stars 8 forks source link

data format #7

Open ghost opened 5 months ago

ghost commented 5 months ago

This is a very innovative and interesting work that has given me a lot of inspiration. I would like to understand the dataset format for the t2i task. Thank you very much for sharing.

saiboxx commented 5 months ago

Hi! Thanks for the interest in our work.

The T2I dataset is defined here: https://github.com/saiboxx/chexray-diffusion/blob/d6d3ffebb07d210be776f9c673743f3f0f2dfd57/cheff/machex.py#L69

A single entry is basically an image and a string.

The text is then tokenized via the BERT tokenizer beforen being passed to the model: https://github.com/saiboxx/chexray-diffusion/blob/d6d3ffebb07d210be776f9c673743f3f0f2dfd57/cheff/ldm/modules/encoders/modules.py#L101

Did that help you or are you looking for something specific?

Cheers, Tobias