mlabonne / llm-autoeval

Automatically evaluate your LLMs in Google Colab
MIT License
460 stars 77 forks source link

Use Colab notebook with private models #18

Closed burtenshaw closed 3 months ago

burtenshaw commented 4 months ago

It is useful to benchmark private models from the hub, but the runpod instance is not authenticated to the user account.

By including the environment variables for HF_TOKEN and colab.userdata, it's straightforward to authenticate the runpod instance. I adapted the notebook here. I'm also happy to submit a PR.

Btw. Great tool. Thanks for your work.