search

prometheus-eval / prometheus-vision

[ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized score rubric, Prometheus-Vision is a good alternative for human evaluation and GPT-4V evaluation.
https://prometheus-eval.github.io/prometheus-vision/
Apache License 2.0
52 stars 6 forks source link

Python example #3

Open teetone opened 7 months ago

teetone commented 7 months ago

Thanks for the great work! Is there an example of how to run inference for a single template/example using HF Transformers without reading from files?

I see the example here: https://huggingface.co/kaist-ai/prometheus-vision-13b-v1.0 but it's not clear to me what question.jsonl and answer.jsonl should look like.