photosynthesis-team / piq

Measures and metrics for image2image tasks. PyTorch.
Apache License 2.0
1.4k stars 120 forks source link

Metric: CLIP-IQA and CLIP-IQA+ #331

Closed snk4tr closed 1 year ago

snk4tr commented 1 year ago

Is your feature request related to a problem? Please describe. CLIP-IQA and CLIP-IQA+ no-reference metrics were introduced in recent publication by Wang et.al.. This metric is interesting because it shows relatively good generalizability and low computational cost (for NN-based metric).

Describe the solution you'd like Add a class or a function with both CLIP-IQA and CLIP-IQA+ metrics.

Additional context Official implementation is available. However, the code has lots of dependencies and needs to be minimized.

snk4tr commented 1 year ago

The main reason to implement CLIP-IQA here is to give a user an opportunity to use the metric without bringing additional dependencies (mmcv/mmedit) to the project.

Note that CLIP-IQA+ won't be implemented here because CLIP weights were fine-tuned with mmcv and hence cannot be loaded and run without it.

Note that the values of this implementation correspond to the values produced by the official CLIP-IQA implementation. SRCC scores of evaluations on public benchmarks may mismatch the ones listed in the paper. We consider official code and weights to be an ultimate source of truth and hence stick with it.