search

Ram81 / goat-bench

60 stars 8 forks source link

How to run the evaluation? #4

Closed TSUNAMIII-7 closed 5 months ago

TSUNAMIII-7 commented 7 months ago

Hi! Thanks for your fantastic work. I just want to run the evaluation part to see how the benchmark work for GOAT task. Is it possible to offer the checkpoint, and the caches which have been mentioned at the config documents (object_cache, image_cache, language_cache)?

Ram81 commented 5 months ago

Hi @TSUNAMIII-7, we have updated the instructions for evaluation here. In addition, the cached embeddings and checkpoints are released here .

TSUNAMIII-7 commented 5 months ago

Thanks for your reply. It really helps!