search

zeno-ml / zeno-evals

Visualize OpenAI Evals with Zeno
MIT License
23 stars 1 forks source link

Error running Evals after installation - pydantic library #1

Open shyam-krish opened 4 months ago

shyam-krish commented 4 months ago

I used OpenAI Evals to run an eval and get a jsonl report. I then pip installed zeno-evals to view the report. But when I went back to run another eval I got the following error:

ImportError: cannot import name 'TypeAdapter' from 'pydantic' (/proj-evals/.env/lib/python3.9/site-packages/pydantic/__init__.cpython-39-darwin.so)

It seems that installing zeno-evals changed some versioning of this pydantic library that was breaking evals so I had to do pip install --upgrade pydantic. This fixed evals but broke zeno-evals.

OS macOS 14.1.1

Python version v3.9.6

JuanmaMenendez commented 4 months ago

@shyam-krish I am having a similar problem. It looks like there is an issue with the pydantic version used for both libraries.

In my case the error is:

"pydantic.errors.PydanticUserError: A non-annotated attribute was detected: calculate_histogram_metrics = True. All model fields require a type annotation; if calculate_histogram_metrics is not meant to be a field, you may be able to resolve this error by annotating it as a ClassVar or updating model_config['ignored_types']."

Any solution ?

shyam-krish commented 4 months ago

@JuanmaMenendez I ended up working around it by creating a different virtual env and pip installed zeno-evals there. I just switch back and forth between the venvs when using evals and zeno evals