search

hdkim01 / DAFT-GAN

pytorch implementation of the paper "DAFT-GAN: Dual Affine Transformation Generative Adversarial Network for Text-Guided Image Inpainting" at ACM Multimedia 2024
1 stars 0 forks source link

dataset #1

Open xhsjwiw opened 3 weeks ago

xhsjwiw commented 3 weeks ago

If I want to train your model on my own dataset, how should I construct the dataset file? And how should the images and their corresponding content be organized in the folders?

hdkim01 commented 3 weeks ago

Hi, @xhsjwiw

First I highly reccomend checking the file "dataset.py", you should adjust the path in the "TextDataset" Class function to match your own data structure.

Using the "flower" and "text_flower" directories, you should create folders for the pickle data along with the "train", "test", and "caption.pickle" files.

data_dir ├── flower (or CUB, COCO) │ └── jpg │ ├── image_00001.jpg │ ├── image_00002.jpg │ └── ... ├── text_flower (or CUB, COCO) │ ├── class_00001 │ │ └── image_06734.txt # 10 prompts for image_06734 │ └── class_00002 │ └── ... ├── train │ ├── filenames.pickle │ └── class_info.pickle ├── test │ ├── filenames.pickle │ └── class_info.pickle └── caption.pickle

xhsjwiw commented 3 weeks ago

Thank you for answering my previous questions. I’ve encountered a new issue. After constructing my dataset, I noticed in the training code there are lines for img_encoder_path = './DAMSMencoders/cub-200-2011/image_encoder200.pth' and text_encoder_path = './DAMSMencoders/cub-200-2011/text_encoder200.pth'. However, when I checked these files, they are empty. I would like to ask how these pre-trained weights are obtained. Thank you very much for your help.

hdkim01 commented 3 weeks ago

You can download pre-trained DAMSM(CUB and COCO) here https://github.com/taoxugit/AttnGAN?tab=readme-ov-file

Or, can also train on your own dataset using pretrain_DAMSM.py