Algolzw / daclip-uir

[ICLR 2024] Controlling Vision-Language Models for Universal Image Restoration. 5th place in the NTIRE 2024 Restore Any Image Model in the Wild Challenge.
https://algolzw.github.io/daclip-uir
MIT License
638 stars 30 forks source link

some question about paper #24

Closed cslvjt closed 8 months ago

cslvjt commented 8 months ago

Thank you for your excellent work. I have a question about the content of your paper. What does gradient flow(in Fig.2a) mean and what does it do? I can't find its meaning elsewhere in the paper.

cslvjt commented 8 months ago

image

As the paper says, $e_d^{I}$ represent degenerate types. Then when you use the degradation type as a conditional constraint, the model should generate a degraded image.

Algolzw commented 8 months ago

Hi!

The gradient flow means the gradient backpropagation path (thanks for pointing it out, we will make it clear in the paper). Moreover, the degenerate type is used to specify image restoration tasks and we use the content embedding (which matches the clean caption) to guide the model to generate clean images.

cslvjt commented 8 months ago

Thank you