truera / trulens

Evaluation and Tracking for LLM Experiments
https://www.trulens.org/
MIT License
2.16k stars 188 forks source link

'pydantic' has no attribute 'model_validator' #730

Closed andysingal closed 10 months ago

andysingal commented 10 months ago

I tried :

pip install -U trulens_eval==0.19.1 -q

import numpy as np
from trulens_eval import Feedback, Tru, TruLlama
from trulens_eval.feedback import Groundedness
from trulens_eval.feedback.provider.openai import OpenAI

tru = Tru()
# Initialize provider class
openai = OpenAI()

grounded = Groundedness(groundedness_provider=OpenAI())

# Define a groundedness feedback function
f_groundedness = Feedback(grounded.groundedness_measure_with_cot_reasons).on(
    TruLlama.select_source_nodes().node.text.collect()
    ).on_output(
    ).aggregate(grounded.grounded_statements_aggregator)

# Question/answer relevance between overall question and answer.
f_qa_relevance = Feedback(openai.relevance).on_input_output()

# Question/statement relevance between question and each context chunk.
f_qs_relevance = Feedback(openai.qs_relevance).on_input().on(
    TruLlama.select_source_nodes().node.text
    ).aggregate(np.mean)

ERROR:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-51-8dfa756af2d5>](https://localhost:8080/#) in <cell line: 2>()
      1 import numpy as np
----> 2 from trulens_eval import Feedback, Tru, TruLlama
      3 from trulens_eval.feedback import Groundedness
      4 from trulens_eval.feedback.provider.openai import OpenAI
      5 

5 frames
[/usr/local/lib/python3.10/dist-packages/trulens_eval/utils/serial.py](https://localhost:8080/#) in Lens()
    554     path: Tuple[Step, ...]
    555 
--> 556     @pydantic.model_validator(mode='wrap')
    557     @classmethod
    558     def validate_from_string(cls, obj, handler):

AttributeError: module 'pydantic' has no attribute 'model_validator'
jimwhite commented 10 months ago

Perhaps you have v1 (instead of v2) of Pydantic installed? Were there any error/warning messages reported by pip? You can check which you have: pip show pydantic

https://github.com/truera/trulens/blob/27b664c8a572d8f72356fa72e326708d57b1fbc0/trulens_eval/trulens_eval/requirements.txt#L10

andysingal commented 10 months ago

Perhaps you have v1 (instead of v2) of Pydantic installed? Were there any error/warning messages reported by pip? You can check which you have: pip show pydantic

https://github.com/truera/trulens/blob/27b664c8a572d8f72356fa72e326708d57b1fbc0/trulens_eval/trulens_eval/requirements.txt#L10

I am running on google colab,

Name: pydantic
Version: 1.10.13
Summary: Data validation and settings management using python type hints
Home-page: https://github.com/pydantic/pydantic
Author: Samuel Colvin
Author-email: [s@muelcolvin.com](mailto:s@muelcolvin.com)
License: MIT
Location: /usr/local/lib/python3.10/dist-packages
Requires: typing-extensions
Required-by: confection, inflect, lida, llmx, spacy, thinc
jimwhite commented 10 months ago

You should be able to update the package with something like:

%pip install -qU trulens_eval pydantic

I've run https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.3.0/trulens_eval/examples/vector-dbs/llama_index/llama_example.ipynb on Colab successfully by adding:

%pip install -qU trulens_eval pydantic fastapi kaleido python-multipart uvicorn cohere openai tiktoken "llama-index"

You can see it here: https://gist.github.com/jimwhite/ec3cb445b04a456f7ccc7ede4a14220e

There are some errors later having to do with leftover Pydantic V1 calls in the notebook because TruLens hasn't fully migrated everything to V2 yet.

andysingal commented 10 months ago

You should be able to update the package with something like:

%pip install -qU trulens_eval pydantic

I've run https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.3.0/trulens_eval/examples/vector-dbs/llama_index/llama_example.ipynb on Colab successfully by adding:

%pip install -qU trulens_eval pydantic fastapi kaleido python-multipart uvicorn cohere openai tiktoken "llama-index"

You can see it here: https://gist.github.com/jimwhite/ec3cb445b04a456f7ccc7ede4a14220e

There are some errors later having to do with leftover Pydantic V1 calls in the notebook because TruLens hasn't fully migrated everything to V2 yet.

Thank you very much it works now :) . Will be adding to my medium article. Cheers :)

satya77 commented 10 months ago

This issue still exists. Even after updating the packages I still get this error.

satya77 commented 10 months ago

Ok I figured out why, i was trying to use an older version of LangChain and it was messing things up.