search

GoogleCloudPlatform / vertex-ai-samples

Notebooks, code samples, sample apps, and other resources that demonstrate how to use, develop and manage machine learning and generative AI workflows using Google Cloud Vertex AI.
https://cloud.google.com/vertex-ai
Apache License 2.0
48 stars 4 forks source link

Stable Diffusion Subject Fidelity(DINO) and Prompt Fidelity(COCA) Calculation #3302

Closed Sundragon1993 closed 2 months ago

Sundragon1993 commented 3 months ago

Dear team, thank you for the comprehensive benchmark of Stable diffusion fine-tune approaches.

Could you give the procedure or snippet of code to calculate the Subject Fidelity(DINO) and Prompt Fidelity (COCA) which were shown in this file: https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/vertex_model_garden/benchmarking_reports/stable_diffusion_v1-5_benchmarking_report.md ?
I've been searching for these keywords but have not had success yet.

weigary commented 2 months ago

Hi,

Thanks for reading the benchmark!

Unfortunately the DINO and COCA calculation algorithms are from an internal implementation which has not yet open sourced. But I believe we can do something similar by following the two papers. There might also be some open-sourced similar algorithms available.

[1] (DINO)](https://arxiv.org/abs/2104.14294) [2] prompt fidelity scores (CoCa),