search

truera / trulens

Evaluation and Tracking for LLM Experiments
https://www.trulens.org/
MIT License
2.11k stars 182 forks source link

【Error】NameError: Fields must not use names with leading underscores #545

Closed wqw547243068 closed 11 months ago

wqw547243068 commented 11 months ago

When I follow the demo in langchain_quickstart.ipynb

Error occurs at this line 

from IPython.display import JSON
# Imports main tools:
from trulens_eval import TruChain, Feedback, Huggingface, Tru # Error here
from trulens_eval.schema import FeedbackResult
tru = Tru()

Full error message is below:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[26], line 4
      1 from IPython.display import JSON
      3 # Imports main tools:
----> 4 from trulens_eval import TruChain, Feedback, Huggingface, Tru
      5 from trulens_eval.schema import FeedbackResult
      6 tru = Tru()

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/trulens_eval/__init__.py:83
      1 """
      2 # Trulens-eval LLM Evaluation Library
      3 
   (...)
     78 
     79 """
     81 __version__ = "0.17.0"
---> 83 from trulens_eval.feedback import Bedrock
     84 from trulens_eval.feedback import Feedback
     85 from trulens_eval.feedback import Huggingface

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/trulens_eval/feedback/__init__.py:14
     11 AggCallable = Callable[[Iterable[float]], float]
     13 # Specific feedback functions:
---> 14 from trulens_eval.feedback.embeddings import Embeddings
     15 # Main class holding and running feedback functions:
     16 from trulens_eval.feedback.feedback import Feedback

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/trulens_eval/feedback/embeddings.py:8
      5 from pydantic import PrivateAttr
      7 from trulens_eval.utils.imports import REQUIREMENT_SKLEARN
----> 8 from trulens_eval.utils.pyschema import WithClassInfo
      9 from trulens_eval.utils.serial import SerialModel
     12 class Embeddings(SerialModel, WithClassInfo):

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/trulens_eval/utils/pyschema.py:588
    584 # Key of structure where class information is stored.
    585 CLASS_INFO = "__tru_class_info"
--> 588 class WithClassInfo(pydantic.BaseModel):
    589     """
    590     Mixin to track class information to aid in querying serialized components
    591     without having to load them.
    592     """
    594     # Using this odd key to not pollute attribute names in whatever class we mix
    595     # this into. Should be the same as CLASS_INFO.

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:104, in __new__(mcs, cls_name, bases, namespace, __pydantic_generic_metadata__, __pydantic_reset_parent_namespace__, **kwargs)

File ~/miniconda3/envs/py310/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py:345, in inspect_namespace(namespace, ignored_types, base_class_vars, base_class_fields)

NameError: Fields must not use names with leading underscores; e.g., use 'WithClassInfo__tru_class_info' instead of '_WithClassInfo__tru_class_info'.
joshreini1 commented 11 months ago

Hi @wqw547243068 - I'm having trouble repro'ing. Can you share your pip list/environment details?

piotrm0 commented 11 months ago

Might be related to pydantic version. Make sure you are running with pydantic < 2.

joshreini1 commented 11 months ago

Closing for now. Please re-open if issue not resolved.

jonmach commented 10 months ago

Hi @wqw547243068 - I'm having trouble repro'ing. Can you share your pip list/environment details?

I have the same problem. I was trying to run the Advanced RAG pipeline (sample code from Deep Learning.ai) . This is failing just on importing the utils file.

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/trulens_eval/utils/pyschema.py:589 585 # Key of structure where class information is stored. 586 CLASS_INFO = "__tru_class_info" --> 589 class WithClassInfo(pydantic.BaseModel): 590 """ 591 Mixin to track class information to aid in querying serialized components 592 without having to load them. 593 """ 595 # Using this odd key to not pollute attribute names in whatever class we mix 596 # this into. Should be the same as CLASS_INFO.

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:92, in new(mcs, cls_name, bases, namespace, pydantic_generic_metadata, __pydantic_reset_parent_namespace__, _create_model_module, **kwargs)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:347, in inspect_namespace(namespace, ignored_types, base_class_vars, base_class_fields)

NameError: Fields must not use names with leading underscores; e.g., use 'WithClassInfo__tru_class_info' instead of '_WithClassInfo__tru_class_info'.

`$ pip list Package Version

aiohttp 3.9.1 aiosignal 1.3.1 aiostream 0.5.2 alembic 1.13.0 altair 5.2.0 annotated-types 0.6.0 anyio 3.7.1 appnope 0.1.3 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 async-lru 2.0.4 attrs 23.1.0 Babel 2.13.1 backoff 2.2.1 beautifulsoup4 4.12.2 bleach 6.1.0 blinker 1.7.0 cachetools 5.3.2 certifi 2023.11.17 cffi 1.16.0 charset-normalizer 3.3.2 click 8.1.7 cohere 4.37 comm 0.2.0 contourpy 1.2.0 cycler 0.12.1 dataclasses-json 0.6.3 datasets 2.15.0 debugpy 1.8.0 decorator 5.1.1 defusedxml 0.7.1 Deprecated 1.2.14 dill 0.3.7 distro 1.8.0 dnspython 2.4.2 entrypoints 0.4 executing 2.0.1 Faker 20.1.0 fastavro 1.9.0 fastjsonschema 2.19.0 favicon 0.7.0 filelock 3.13.1 fonttools 4.46.0 fqdn 1.5.1 frozendict 2.3.10 frozenlist 1.4.0 fsspec 2023.10.0 gitdb 4.0.11 GitPython 3.1.40 greenlet 3.0.1 h11 0.14.0 htbuilder 0.6.2 html2text 2020.1.16 httpcore 1.0.2 httpx 0.25.2 huggingface-hub 0.19.4 humanize 4.9.0 idna 3.6 importlib-metadata 6.11.0 InstructorEmbedding 1.0.1 ipykernel 6.27.1 ipython 8.18.1 ipywidgets 8.1.1 isoduration 20.11.0 jedi 0.19.1 Jinja2 3.1.2 joblib 1.3.2 json5 0.9.14 jsonpatch 1.33 jsonpointer 2.4 jsonschema 4.20.0 jsonschema-specifications 2023.11.2 jupyter 1.0.0 jupyter_client 8.6.0 jupyter-console 6.6.3 jupyter_core 5.5.0 jupyter-events 0.9.0 jupyter-lsp 2.2.1 jupyter_server 2.11.2 jupyter_server_terminals 0.4.4 jupyterlab 4.0.9 jupyterlab_pygments 0.3.0 jupyterlab_server 2.25.2 jupyterlab-widgets 3.0.9 kaggle 1.5.16 kiwisolver 1.4.5 langchain 0.0.345 langchain-core 0.0.9 langsmith 0.0.69 llama-index 0.9.11.post1 loguru 0.7.2 lxml 4.9.3 Mako 1.3.0 Markdown 3.5.1 markdown-it-py 3.0.0 markdownlit 0.0.7 MarkupSafe 2.1.3 marshmallow 3.20.1 matplotlib 3.8.2 matplotlib-inline 0.1.6 mdurl 0.1.2 merkle-json 1.0.0 millify 0.1.1 mistune 3.0.2 more-itertools 10.1.0 mpmath 1.3.0 multidict 6.0.4 multiprocess 0.70.15 munch 4.0.0 mypy-extensions 1.0.0 nbclient 0.9.0 nbconvert 7.12.0 nbformat 5.9.2 nest-asyncio 1.5.8 networkx 3.2.1 nltk 3.8.1 notebook 7.0.6 notebook_shim 0.2.3 numpy 1.26.2 openai 1.3.7 overrides 7.4.0 packaging 23.2 pandas 2.1.3 pandocfilters 1.5.0 parso 0.8.3 pexpect 4.9.0 Pillow 10.1.0 pinecone-client 2.2.4 pip 23.3.1 platformdirs 4.1.0 prometheus-client 0.19.0 prompt-toolkit 3.0.41 protobuf 4.25.1 psutil 5.9.6 ptyprocess 0.7.0 pure-eval 0.2.2 pyarrow 14.0.1 pyarrow-hotfix 0.6 pycparser 2.21 pydantic 1.10.13 pydantic_core 2.14.5 pydeck 0.8.1b0 Pygments 2.17.2 pymdown-extensions 10.5 pyparsing 3.1.1 pypdf 3.17.1 python-dateutil 2.8.2 python-decouple 3.8 python-dotenv 1.0.0 python-json-logger 2.0.7 python-slugify 8.0.1 pytz 2023.3.post1 PyYAML 6.0.1 pyzmq 25.1.1 qtconsole 5.5.1 QtPy 2.4.1 referencing 0.31.1 regex 2023.10.3 requests 2.31.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.7.0 rpds-py 0.13.2 safetensors 0.4.1 scikit-learn 1.3.2 scipy 1.11.4 Send2Trash 1.8.2 sentence-transformers 2.2.2 sentencepiece 0.1.99 setuptools 65.5.0 six 1.16.0 slack-bolt 1.18.1 slack-sdk 3.26.1 smmap 5.0.1 sniffio 1.3.0 soupsieve 2.5 SQLAlchemy 2.0.23 st-annotated-text 4.0.1 stack-data 0.6.3 streamlit 1.29.0 streamlit-aggrid 0.3.4.post3 streamlit-camera-input-live 0.2.0 streamlit-card 0.0.61 streamlit-embedcode 0.1.2 streamlit-extras 0.3.5 streamlit-faker 0.0.3 streamlit-image-coordinates 0.1.6 streamlit-javascript 0.1.5 streamlit-keyup 0.2.0 streamlit-toggle-switch 1.0.2 streamlit-vertical-slider 1.0.2 sympy 1.12 tenacity 8.2.3 terminado 0.18.0 text-unidecode 1.3 threadpoolctl 3.2.0 tiktoken 0.5.2 tinycss2 1.2.1 tokenizers 0.15.0 toml 0.10.2 toolz 0.12.0 torch 2.1.1 torchvision 0.16.1 tornado 6.4 tqdm 4.66.1 traitlets 5.14.0 transformers 4.35.2 trulens-eval 0.18.2 types-python-dateutil 2.8.19.14 typing_extensions 4.8.0 typing-inspect 0.9.0 tzdata 2023.3 tzlocal 5.2 uri-template 1.3.0 urllib3 2.1.0 validators 0.22.0 wcwidth 0.2.12 webcolors 1.13 webencodings 0.5.1 websocket-client 1.7.0 widgetsnbextension 4.0.9 wrapt 1.16.0 xxhash 3.4.1 yarl 1.9.3 you-get 0.4.1650 zipp 3.17.0`

Takher commented 10 months ago

If you're using a notebook and have just installed trulens_eval, restarting the kernel can fix unusual errors -- try that before downgrading packages manually.

jonmach commented 10 months ago

Thanks. Restarting the kernel didn't help, but I blew away all environments and reinstalled from scratch. Now seems to go a little further.

SamyPal commented 10 months ago

Same here. I also have pydantic 1.10.11 which is <2 as recommended above. Restarting also did not help my case. I am getting the below error:

line 1 ----> 1 from trulens_eval import Tru 2 tru = Tru() 4 tru.reset_database()

File ~/.local/lib/python3.10/site-packages/trulens_eval/init.py:97 95 from trulens_eval.tru_custom_app import instrument 96 from trulens_eval.tru_custom_app import TruCustomApp ---> 97 from trulens_eval.tru_llama import TruLlama 98 from trulens_eval.utils.threading import TP 100 all = [ 101 'Tru', 102 'TruBasicApp', (...) 115 'TP' 116 ]

File ~/.local/lib/python3.10/site-packages/trulens_eval/tru_llama.py:44 41 from llama_index.schema import BaseComponent 43 # LLMs ---> 44 from llama_index.llms.base import LLM # subtype of BaseComponent 46 # misc 47 from llama_index.indices.query.base import BaseQueryEngine

ImportError: cannot import name 'LLM' from 'llama_index.llms.base' (/home/jovyan/.local/lib/python3.10/site-packages/llama_index/llms/base.py)

Priye-Ranjan commented 10 months ago

from trulens_eval import Tru tru = Tru()

Getting the below error on executing the above code- Traceback (most recent call last): File "/Users/local/Desktop/chatbot/rag.py", line 1, in from trulens_eval import Tru File "/Users/local/Desktop/chatbot/myenv/lib/python3.10/site-packages/trulens_eval/init.py", line 97, in from trulens_eval.tru_llama import TruLlama File "/Users/local/Desktop/chatbot/myenv/lib/python3.10/site-packages/trulens_eval/tru_llama.py", line 44, in from llama_index.llms.base import LLM # subtype of BaseComponent ImportError: cannot import name 'LLM' from 'llama_index.llms.base

julfr commented 8 months ago

Same here. I also have pydantic 1.10.11 which is <2 as recommended above. Restarting also did not help my case. I am getting the below error:

line 1 ----> 1 from trulens_eval import Tru 2 tru = Tru() 4 tru.reset_database()

File ~/.local/lib/python3.10/site-packages/trulens_eval/init.py:97 95 from trulens_eval.tru_custom_app import instrument 96 from trulens_eval.tru_custom_app import TruCustomApp ---> 97 from trulens_eval.tru_llama import TruLlama 98 from trulens_eval.utils.threading import TP 100 all = [ 101 'Tru', 102 'TruBasicApp', (...) 115 'TP' 116 ]

File ~/.local/lib/python3.10/site-packages/trulens_eval/tru_llama.py:44 41 from llama_index.schema import BaseComponent 43 # LLMs ---> 44 from llama_index.llms.base import LLM # subtype of BaseComponent 46 # misc 47 from llama_index.indices.query.base import BaseQueryEngine

ImportError: cannot import name 'LLM' from 'llama_index.llms.base' (/home/jovyan/.local/lib/python3.10/site-packages/llama_index/llms/base.py)

I have the same problem, have you solved it?

Priye-Ranjan commented 8 months ago

Same here. I also have pydantic 1.10.11 which is <2 as recommended above. Restarting also did not help my case. I am getting the below error: line 1 ----> 1 from trulens_eval import Tru 2 tru = Tru() 4 tru.reset_database() File ~/.local/lib/python3.10/site-packages/trulens_eval/init.py:97 95 from trulens_eval.tru_custom_app import instrument 96 from trulens_eval.tru_custom_app import TruCustomApp ---> 97 from trulens_eval.tru_llama import TruLlama 98 from trulens_eval.utils.threading import TP 100 all = [ 101 'Tru', 102 'TruBasicApp', (...) 115 'TP' 116 ] File ~/.local/lib/python3.10/site-packages/trulens_eval/tru_llama.py:44 41 from llama_index.schema import BaseComponent 43 # LLMs ---> 44 from llama_index.llms.base import LLM # subtype of BaseComponent 46 # misc 47 from llama_index.indices.query.base import BaseQueryEngine ImportError: cannot import name 'LLM' from 'llama_index.llms.base' (/home/jovyan/.local/lib/python3.10/site-packages/llama_index/llms/base.py)

I have the same problem, have you solved it?

I'm not sure whether it was an issue in Trulens library or a Python version issue but somehow it got resolved when I started from scratch by creating a new virtual env and it worked for me.