search

openai / evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Other
14.67k stars 2.57k forks source link

Multiple evals not found #1379

Open SUMEETRM opened 11 months ago

SUMEETRM commented 11 months ago

Describe the bug

When running evals such as the steganography eval, I get: AssertionError: Eval steganography.yaml not found

To Reproduce

Install and set up everything according to the documentation oaieval gpt-3.5-turbo steganography

Code snippets

No response

OS

macOS

Python version

Python 3.9

Library version

main

pskl commented 11 months ago

Hello, you need to specify 2 completion functions to make this eval work: oaieval gpt-3.5-turbo,gpt-3.5-turbo steganography

It works for me at least.

SUMEETRM commented 11 months ago

Hello, you need to specify 2 completion functions to make this eval work: oaieval gpt-3.5-turbo,gpt-3.5-turbo steganography

It works for me at least.

Unfortunately that still doesn't work for me. Did you change anything else too?

pskl commented 11 months ago

Did you pull the git lfs files? Is your main branch up to date?

SUMEETRM commented 11 months ago

I now get this issue after it starts running: UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 7: ordinal not in range(256)

pskl commented 10 months ago

That I have no clue about, I'd say reinstall, repull the files, etc