deblurring task isn't giving me deblurred results, is this expected?

[Njasa2k]

[canqin001]

Thank you for your attempt. I just tested your example and tried different "control strength". It gets some better results. I guess this input image has different types of blurry compared with our synthetic blurred training data. We will deeply investigate it and may update our model to make it better.

[Howard-Hsiao]

I've encountered the same problem as well. It appears that nearly all the images that undergo deblurring retain traces of the painting's texture or mosaic-like patterns. Even when I followed the sample provided in the README.md, I couldn't achieve the desired outcome. Are there any additional suggestions or tips that I could explore?

[canqin001]

This is weird. I guess this is caused by our limited training data which is synthesized from clean images by apply Gaussian blur kernels. Therefore, there could the domain gap between the real-world blurred images and synthetic ones. We may need to generate more training data to re-train the model. Sorry for that and thank you for letting us know. I hope this can be addressed in the next version UniControl.

[Howard-Hsiao]

Thanks for your reply. I also wanted to ask what does the condition extraction option do for the deblur task?

[canqin001]

It turns the clean image to the blurred one by applying Gaussian blur kernel.

[Howard-Hsiao]

Greetings,

I have observed the effective performance of deblur and colorization tasks within the zero-shot context. My curiosity pertains to the alignment of the model provided in this repository with the aforementioned context. This prompts the question of whether the model has been trained without exposure to any paired data associated with the deblur and colorization tasks. This uncertainty arises from my observation that the multigen20m dataset employed in your work encompasses data relevant to both of these tasks. Your clarification on this matter would be greatly appreciated.

[canqin001]

Thank you for this question. The paper version model (three months ago) is only pre-trained by 9 tasks and the results demonstrated in the paper are zero-shot. This newly released 12-task checkpoint includes the supervisedly training of blurring, colorization and inpainting to ensure the stable performance on these tasks.

[Howard-Hsiao]

Thanks for your quick reply. Could you please clarify whether the model employed in the Huggingface space is the 9-task checkpoint or the 12-task checkpoint?

[canqin001]

The HF space one is 12-task.