search

DQiaole / ZITS_inpainting

[CVPR 2022] Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding
Apache License 2.0
336 stars 36 forks source link

Questions to get started #84

Open benijohn opened 3 weeks ago

benijohn commented 3 weeks ago

Hello, thank you for sharing your work. We want to adapt this to our own data and we are having issues even getting started.

I have a few questions on how to move forward.

  1. On step 4 "prepare wireframes". What should be the input images? We want to use this on elevation map data. But we are also hoping to make sure everything working as expected before trying to train it further on our data.
  2. For Eval. I downloaded everything listed in your google drive and put them in folder named ckpt at the root of the project. I tried to Test on 256 images by running the command: python FTR_inference.py --path ./ckpt/zits_places2 --config_file ./config_list/config_ZITS_places2.yml --GPU_ids '0' Which gave me the error:

FileNotFoundError: [Errno 2] No such file or directory: '/home/wmlce/places365_standard/places2_all/test_sub_list.txt' so I realize I need to update the config paths. But I am not sure what test_sub_list.txt should contain? Should VAL_FLIST and TEST_FLIST really be the same lists? Do I also need to provide the TRAIN_FLIST to test the inference? Do I need the whole places dataset?

3.For the irregular_mask_list and coco_mask_list do I need both? If so, do they just need to include the paths to the files? Or do the lists have to match the size of the TRAIN sets and TEST sets respectively?

  1. Where do I find the lama_places2/InpaintingModel_gen and dis.pth? Do I need to have these? Will I need it for training?
  2. I also tried running a single inference by using the command: python single_image_test.py --path=./best_transformer_places2.pth --config_file=config_list/config_ZITS_places2.yml --GPU_ids='0' --img_path=elevation.png --mask_path=mask.png --save_path=./

FileExistsError: [Errno 17] File exists: './best_transformer_places2.pth' but I see that the exists_ok flag is set to True. Any idea why it would be failing here?

Sorry for so many questions. It seems that everything I have tried results in an error and I am not sure what I am doing wrong. Thanks for any input you can offer.

Cheers, Benjamin

DQiaole commented 1 week ago

Hi, sorry for the late reply:

  1. the input images for wireframes extraction is the original images that you want to do inpainting.
  2. here you can set VAL_FLIST and TEST_FLIST be the same lists if you didnot have another vlaidation set.
  3. they just need to include the paths to the file. the size can be different, because we will keep repeating when there are not enough masks.
  4. yes, you should train a model of lama first: https://github.com/DQiaole/ZITS_inpainting?tab=readme-ov-file#training-lama-first
  5. the path is the directory of checkpoints, not the model path. You can put all the pretrained models as follows: https://github.com/DQiaole/ZITS_inpainting/blob/6f4a47bb03bfb82a4860fc55ae0719ab0db8f6eb/config_list/config_ZITS_places2.yml#L13-L17 , and set the path to ./ckpt/lama_places2.