Can this method Achieve face-mask realistic inpainting ?

[francqz31]

Hello , How are you Mr. author ! thank you for being active and answering all the questions!

DiffPIR is so good at Face Depixelization or also what can be referred to as Restoration and it is also really good at simple inpainting tasks with animal pics. I was wondering if it can be trained or solely fine-tuned to do Face-mask inpainting like (https://github.com/andreas128/RePaint) and even if it can get better results than Repaint after the training or the fine-tuning ? and if so , how can I do it ?

for reference, so it can do tasks like that and replace the mask with very accurate face

[yuanzhi-zhu]

Hi @francqz31,

Both DiffPIR and RePaint are training-free methods, which means no further training or fine-tuning is required.

Specifically, in DiffPIR we can load any custom mask from an image (you have to draw the mask manually) through the load_mask option https://github.com/yuanzhi-zhu/DiffPIR/blob/main/main_ddpir_inpainting.py#L167. (DiffPIR can be faster than RePaint without sacrificing quality, but no better results are guaranteed 🥲)

To conclude, DiffPIR can solve custom inpainting tasks by giving the image and mask pair just like in RePaint. And the generated content in the masked region depends on the prior knowledge in the pre-trained diffusion model.

P.S: However, exercise caution with the outcome you receive👿!!

[francqz31]

@yuanzhi-zhu Thanks so much for the clarification! Yes Repaint is so slow and expensive to run ,DiffPIR is amazing. I made a colab or an Interactive demo Here https://colab.research.google.com/drive/1fc36_MNmcsNANLzTMTtKI2JgNunF09i2 you can use it or put it in the readme for people if you want. You can also download it from here https://fex.net/s/z9nr6cp and save the notebook on your account and then upload it to the repo.

2- second question, How can i draw a mask manually ? like I think after I set "load_mask = True" i should make a directory called "gt_keep_masks" and then put the drawn mask inside of it , but how can i draw a mask manually ?

P.S: that pic above with the beard made me laugh so hard😂,

[yuanzhi-zhu]

@francqz31 Thank you for your notebook, I will add a demo in the next major update (if any).

Unfortunately, I do not know how to draw a mask efficiently. (I just brush the image with black and white