search

McGill-NLP / webllama

Llama-3 agents that can browse the web by following instructions and talking to you
https://webllama.github.io
MIT License
1.35k stars 101 forks source link

Comparison with Llama3 #6

Closed elmoBG8 closed 5 months ago

elmoBG8 commented 6 months ago

I see you compared your version with other models, but what about the version from which you derived it? Thanks

xhluca commented 6 months ago

In the weblinx paper, we have found that non-finetuned models are not as good as finetuned models, so it is unlikely that Llama-3-8b-Instruct surpasses Llama-3-8B-Web. However, if you think it's worth adding the results, it should be fairly straightforward to run the eval script with minor changes to the config: https://github.com/McGill-NLP/webllama/tree/main/modeling#run-llama-on-evaluation-splits