search

Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.63k stars 168 forks source link

How to evaluate SPHINX on refCOCO? #130

Closed CrazyLenmon closed 6 months ago

CrazyLenmon commented 6 months ago

Thanks for your great work! I'm wondering how to evaluate your model on refCOCO. I see that some documents and codes you didn't release including the utils and scripts in accessory/eval_mm/. When do you plan to release? And is it possible for me to evaluate only on one dataset? I am looking forward to your reply. image

Artanic30 commented 6 months ago

Sorry for the mistakes. The missing files are updated in https://github.com/Alpha-VLLM/LLaMA2-Accessory/pull/131. We update the instruction for evaluation on one dataset in README.md.

CrazyLenmon commented 6 months ago

Sorry for the mistakes. The missing files are updated in #131. We update the instruction for evaluation on one dataset in README.md.

There are still some funcitons missing. It doesn't work. Could you please check it again? image

Artanic30 commented 6 months ago

The missing file is updated in https://github.com/Alpha-VLLM/LLaMA2-Accessory/pull/133.

CrazyLenmon commented 6 months ago

The missing file is updated in #133.

Thank you. And I noticed that there are some mistakes in codes. I suggest that you may check the codes. Some mistakes such as "pretrined_path" and like in this image. You may try to run the version you commited to find these codes mistakes. Anyway, thank you for your great effort. image

Artanic30 commented 6 months ago

Thanks for pointing out the code version issue, we will fix the code as soon as possible.