truera / trulens

Evaluation and Tracking for LLM Experiments
https://www.trulens.org/
MIT License
2.14k stars 186 forks source link

Pydantic issue? #621

Closed nicolascepeda closed 10 months ago

nicolascepeda commented 11 months ago

Hi Guys, thanks for the awesome work! TrueLens is a Gem!

I am currently doing the setup on my project but right after doing the pip install of true_eval and run the code I get following exception:

NameError: Fields must not use names with leading underscores; e.g., use 'WithClassInfo__tru_class_info' instead of '_WithClassInfo__tru_class_info'.

I assume it to be a problem with the pydantic dependency, in the requirements.txt there is no pydantic version specified. Do you have seen this error in the past? Any idea what is the current pydantic version being supported?

Thanks a lot

File [/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:589](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:589)
    [585](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:585) # Key of structure where class information is stored.
    [586](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:586) CLASS_INFO = "__tru_class_info"
--> [589](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:589) class WithClassInfo(pydantic.BaseModel):
    [590](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:590)     """
    [591](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:591)     Mixin to track class information to aid in querying serialized components
    [592](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:592)     without having to load them.
    [593](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:593)     """
    [595](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:595)     # Using this odd key to not pollute attribute names in whatever class we mix
    [596](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:596)     # this into. Should be the same as CLASS_INFO.

File [/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:98](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:98), in __new__(mcs, cls_name, bases, namespace, __pydantic_generic_metadata__, __pydantic_reset_parent_namespace__, **kwargs)

File [/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:322](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/project/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:322), in inspect_namespace(namespace, ignored_types, base_class_vars, base_class_fields)

NameError: Fields must not use names with leading underscores; e.g., use 'WithClassInfo__tru_class_info' instead of '_WithClassInfo__tru_class_info'.
joshreini1 commented 11 months ago

Thanks for the praise @nicolascepeda !!

@piotrm0 is taking on a bunch of the pydantic work right now so I'll assign to him.

marycampus commented 11 months ago

hey - i have the same issue. Trulens is using an old version of pydantic which isn't compatible with the latest. unfortunately this is causing an unresolvable dependency conflict in my project - Because no versions of trulens-eval match >0.18.1,<0.19.0 and trulens-eval (0.18.1) depends on pydantic (>=1.10.7,<2), trulens-eval (>=0.18.1,<0.19.0) requires pydantic (>=1.10.7,<2). So, because ce-demo depends on both pydantic (2.4.2) and trulens-eval (^0.18.1), version solving failed.

piotrm0 commented 11 months ago

Our latest release does not work with pydantic >= 2. This work is in progress and should be ready for release next week.

marycampus commented 11 months ago

thank you! I will circle back next week. Really excited to try this out!

mnicstruwig commented 11 months ago

To add a voice to the choir, we're also waiting for Pydantic v2 support (or at least some mechanism to allow Pydantic v2 to function in the environment, since we depend on projects that now require Pydantic >=2).

Looking forward to the release, hope it ships soon!

joshreini1 commented 10 months ago

Hey @mnicstruwig @marycampus @nicolascepeda ! Pydantic v2 support is released! Please upgrade to trulens_eval==0.19.1

Thanks!