search

aehrc / cvt2distilgpt2

Improving Chest X-Ray Report Generation by Leveraging Warm-Starting
GNU General Public License v3.0
61 stars 5 forks source link

could I utilize this work on other datasets? #8

Open jessie-chen99 opened 1 year ago

jessie-chen99 commented 1 year ago

hi thanks for your great work.

i am working on brain tumor segmentation, could i use this model and its ckpt to generate a useful text from brain image(like image captioning, but in medical domain)?

anicolson commented 1 year ago

Hi Jessie,

To get a model such as this to do what you want, you would have to perform fine-tuning/transfer learning with a dataset of the target task.

A.

kwokchunghim commented 1 year ago

Hi, I am working on an ARRG project and would attempt to finetune your model on a custom dataset. Is it possible for you to include guidelines or instructions on how to finetune your model on other datasets? In particular, is there any special config that we have to modify? Thanks a lot!

anicolson commented 1 year ago

Hi, you would need to make a custom https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset. Then modify the lightning module to feed the torch dataset to the dataloaders. The torch datasets are initialised in the lightning modules setup definition. Look in the config directory for the config files.

Hope this helps, A.