carlini / yet-another-applied-llm-benchmark

A benchmark to evaluate language models on questions I've previously asked them to solve.
GNU General Public License v3.0
875 stars 64 forks source link

minor fix to successfully run a single test case #2

Closed ViswanathaReddyGajjala closed 7 months ago

ViswanathaReddyGajjala commented 7 months ago

I wanted to add a few test cases, but it took me some time to get it working for a sample test case. I hope this change helps others save some time.

I made a few changes to ensure one specific test works on Colab:

  1. I copied the config.json.example file to create a new file named config.json and added the OpenAI key.
  2. I updated the Python version in the code and included the information in the readme.
  3. I tested these changes on Colab, and everything is working correctly.
ViswanathaReddyGajjala commented 7 months ago

Exact commands for reference

After making the changes mentioned above, I executed the following commands on Colab:

!git clone https://github.com/carlini/yet-another-applied-llm-benchmark.git

%cd yet-another-applied-llm-benchmark
!pip install -r requirements.txt
!pip install -r requirements-extra.txt

!PYTHONPATH='.' python tests/print_hello.py 
carlini commented 7 months ago

Do you want to add a subsection "Running from Colab" and include this information? It seems useful.

ViswanathaReddyGajjala commented 7 months ago

Thank you for the suggestion. Instead, I've created a Colab notebook, making it much easier to run. Feel free to share any further suggestions or improvements!