search

liuqk3 / PUT

Paper 'Transformer based Pluralistic Image Completion with Reduced Information Loss' in TPAMI 2024 and 'Reduce Information Loss in Transformers for Pluralistic Image Inpainting' in CVPR2022
MIT License
178 stars 15 forks source link

script inference for a single image #1

Open cahya-wirawan opened 2 years ago

cahya-wirawan commented 2 years ago

Hi,

Thanks for the source code, the result looks great. Btw, do you have a simple script to do the inference for a single image (and mask)? Thanks.

liuqk3 commented 2 years ago

@cahya-wirawan Thanks for your interesting!

I have not implemented the inference function for a single image yet. In fact, you can simply put the image and mask into different directories, and modify the dataloader configuration in the yaml file.

I indeed have the plan to release the codes for the simple usage (just inference) with a single command line. This issue shall be closed at that time. Please keep an eye on this issue and be patient.

cahya-wirawan commented 2 years ago

Hi @liuqk3

Thanks for the response. I am looking forward for your code to do inference of a single image.

Meanwhile, I was able to do inference of a single image (little bit change in your code: https://github.com/cahya-wirawan/PUT/tree/dev), however the result is not very convincing. I run it with following command python scripts/inference.py --name transformer_ffhq --func inference_complet_sample_in_feature_for_evaluation --gpu 0 --batch_size 1, and here is the result: https://enemy.org/~cahya/PUT/RESULT/ What did I do wrong here?

liuqk3 commented 2 years ago

Hi @cahya-wirawan

PUT can produce diverse results, but not all produced results are of high quality. You can try to generate more results. In addition, there are some hyper-parameters for inference function. For example, the top-k. According to my experience, your mask ratio is quite low, it shoule be easy for PUT to complete the missing pixels.

cahya-wirawan commented 2 years ago

Hi @liuqk3 , How can I use the hyper parameter top-k? I can't find it in the script inference.py

liuqk3 commented 2 years ago

Hi @cahya-wirawan , filter_ratio in the inference script controlls the value of K. If it is an integer number and greater than 1, it is the value of K. If it is smaller than 1 and is a float number, (1 - filter_ratio)*512 is the value of K, where 512 is the number of tokens. https://github.com/liuqk3/PUT/blob/56450c90bdd36aee43e28e45ad5a5fdb94aa0ed6/scripts/inference.py#L545

cahya-wirawan commented 2 years ago

Ok, I'll try it. Thanks for the info