Closed rajpr5656 closed 2 months ago
Hey @rajpr5656! Good to see you tackling new challenges with TruLens. Let's dive into this dashboard dilemma together! 🚀
I understand that you're having trouble with the TruLens dashboard's Leaderboard not displaying as expected. Let's try to troubleshoot this issue step by step.
Firstly, it's important to ensure that all the packages are compatible with each other and with Python 3.12.2. Since Python 3.12 introduced several changes, some packages might not yet be fully compatible. You've mentioned aiohttp 3.9.3 and fastapi 0.110.0, which should be checked for compatibility with Python 3.12.2.
Secondly, although you're using TruLens version 0.13.4, it's a good practice to check if there's a newer version available that might have fixed the issue you're encountering. You can update TruLens to the latest version using the command pip install trulens --upgrade
.
Next, ensure all dependencies are correctly installed and up to date. You can do this by running pip install -r requirements.txt
if you have a requirements file or manually updating packages mentioned in your setup like aiohttp and fastapi using pip install aiohttp --upgrade
and pip install fastapi --upgrade
.
Also, ensure that the port 8502 is not already in use on your system. You can check this by using the netstat -aon | findstr 8502
command in the command prompt. If the port is in use, either close the application using it or choose a different port for the TruLens dashboard.
Sometimes, running the command prompt or PowerShell as an administrator can resolve issues related to permissions, especially on Windows 11. So, try running your application as an administrator.
Ensure that your firewall or antivirus is not blocking the application or the specific port you're trying to use. This can often cause unexpected issues.
When running tru.run_dashboard(port=8502)
, carefully check the console output for any errors or warnings that might give you more insight into what's going wrong.
If after following these steps the issue persists, it would be helpful to have more detailed information about the error messages you're receiving and the steps you've already taken to try to resolve the issue. This will help in diagnosing the problem more accurately.
For more information, you can refer to the TruLens codebase and documentation:
I hope this helps in resolving the issue. If you have any more questions or need further clarification, feel free to ask.
Thanks for your reply. Could you please let me know what version of Python should I reinstall to make tru.run_dashboard() to work. I will install recommended version of python. Also, pls recommend associated packages and its version to be installed. I will recreate all.
@rajpr5656 can you share the stack trace?
Could you pls provide steps? I am sorry for asking basic question. If you are asking for sequence of operations of LLM then please find below: Followed below URL https://github.com/kevintsai/Building-and-Evaluating-Advanced-RAG-Applications/blob/main/L1-Advanced_RAG_Pipeline.ipynb
and only performed the following operations of
tru_recorder = get_prebuilt_trulens_recorder(query_engine, app_id="Direct Query Engine")' I copied your util.py code that has feedback codes.
Also, I degraded my python to 3.11.8 Here are my latest python packages: Package Version
aiohttp 3.9.3 aiosignal 1.3.1 aiostream 0.5.2 alembic 1.13.1 altair 4.2.2 annotated-types 0.6.0 anyio 4.3.0 asttokens 2.4.1 attrs 23.2.0 backoff 2.2.1 beautifulsoup4 4.12.3 bleach 6.1.0 blinker 1.7.0 cachetools 5.3.3 certifi 2024.2.2 charset-normalizer 3.3.2 click 8.1.7 cohere 5.1.7 colorama 0.4.6 comm 0.2.2 contourpy 1.2.0 cycler 0.12.1 dataclasses-json 0.6.4 datasets 2.18.0 debugpy 1.8.1 decorator 5.1.1 Deprecated 1.2.14 dill 0.3.8 dirtyjson 1.0.8 distro 1.9.0 dnspython 2.6.1 entrypoints 0.4 executing 2.0.1 Faker 24.4.0 fastavro 1.9.4 favicon 0.7.0 filelock 3.13.3 fonttools 4.50.0 frozendict 2.4.0 frozenlist 1.4.1 fsspec 2024.2.0 gitdb 4.0.11 GitPython 3.1.42 greenlet 3.0.3 h11 0.14.0 htbuilder 0.6.2 html2text 2024.2.26 httpcore 1.0.5 httpx 0.27.0 huggingface-hub 0.22.2 humanize 4.9.0 idna 3.6 importlib_metadata 7.1.0 ipykernel 6.29.4 ipython 8.22.2 ipywidgets 8.1.2 jedi 0.19.1 Jinja2 3.1.3 joblib 1.3.2 jsonpatch 1.33 jsonpointer 2.4 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyter_client 8.6.1 jupyter_core 5.7.2 jupyterlab_widgets 3.0.10 kaggle 1.6.8 kiwisolver 1.4.5 langchain 0.1.13 langchain-community 0.0.29 langchain-core 0.1.36 langchain-text-splitters 0.0.1 langsmith 0.1.38 llama-index 0.10.25 llama-index-agent-openai 0.2.1 llama-index-cli 0.1.11 llama-index-core 0.10.25.post3 llama-index-embeddings-huggingface 0.2.0 llama-index-embeddings-openai 0.1.7 llama-index-indices-managed-llama-cloud 0.1.5 llama-index-legacy 0.9.48 llama-index-llms-openai 0.1.14 llama-index-multi-modal-llms-openai 0.1.4 llama-index-program-openai 0.1.5 llama-index-question-gen-openai 0.1.3 llama-index-readers-file 0.1.13 llama-index-readers-llama-parse 0.1.4 llama-parse 0.4.0 llamaindex-py-client 0.1.15 loguru 0.7.2 lxml 5.1.1 Mako 1.3.2 Markdown 3.6 markdown-it-py 3.0.0 markdownlit 0.0.7 MarkupSafe 2.1.5 marshmallow 3.21.1 matplotlib 3.8.3 matplotlib-inline 0.1.6 mdurl 0.1.2 merkle-json 1.0.0 millify 0.1.1 minijinja 1.0.16 more-itertools 10.2.0 mpmath 1.3.0 multidict 6.0.5 multiprocess 0.70.16 munch 4.0.0 mypy-extensions 1.0.0 nest-asyncio 1.6.0 networkx 3.2.1 nltk 3.8.1 numpy 1.26.4 openai 1.14.3 orjson 3.10.0 packaging 23.2 pandas 2.2.1 parso 0.8.3 pillow 10.2.0 pinecone-client 3.2.2 pip 24.0 platformdirs 4.2.0 prometheus_client 0.20.0 prompt-toolkit 3.0.43 protobuf 4.25.3 psutil 5.9.8 pure-eval 0.2.2 pyarrow 15.0.2 pyarrow-hotfix 0.6 pydantic 2.6.4 pydantic_core 2.16.3 pydeck 0.8.0 Pygments 2.17.2 pymdown-extensions 10.7.1 PyMuPDF 1.24.0 PyMuPDFb 1.24.0 pyparsing 3.1.2 pypdf 4.1.0 python-dateutil 2.9.0.post0 python-decouple 3.8 python-dotenv 1.0.1 python-slugify 8.0.4 pytz 2024.1 pywin32 306 PyYAML 6.0.1 pyzmq 25.1.2 referencing 0.34.0 regex 2023.12.25 requests 2.31.0 rich 13.7.1 rpds-py 0.18.0 safetensors 0.4.2 scikit-learn 1.4.1.post1 scipy 1.12.0 sentence-transformers 2.6.1 sentencepiece 0.2.0 setuptools 65.5.0 six 1.16.0 slack-bolt 1.18.1 slack_sdk 3.27.1 smmap 5.0.1 sniffio 1.3.1 soupsieve 2.5 SQLAlchemy 2.0.29 st-annotated-text 4.0.1 stack-data 0.6.3 streamlit 1.32.2 streamlit-aggrid 1.0.0 streamlit-camera-input-live 0.2.0 streamlit-card 1.0.0 streamlit-embedcode 0.1.2 streamlit-extras 0.4.0 streamlit-faker 0.0.3 streamlit-image-coordinates 0.1.6 streamlit-javascript 0.1.5 streamlit-keyup 0.2.3 streamlit-toggle-switch 1.0.2 streamlit-vertical-slider 2.5.5 striprtf 0.0.26 sympy 1.12 tenacity 8.2.3 text-unidecode 1.3 threadpoolctl 3.4.0 tiktoken 0.6.0 tokenizers 0.15.2 toml 0.10.2 toolz 0.12.1 torch 2.2.2 torchvision 0.17.2 tornado 6.4 tqdm 4.66.2 traitlets 5.14.2 transformers 4.39.2 trulens-eval 0.27.0 types-requests 2.31.0.20240311 typing_extensions 4.10.0 typing-inspect 0.9.0 tzdata 2024.1 tzlocal 5.2 urllib3 2.2.1 utils 1.0.2 validators 0.24.0 watchdog 4.0.0 wcwidth 0.2.13 webencodings 0.5.1 widgetsnbextension 4.0.10 win32-setctime 1.1.0 wrapt 1.16.0 xxhash 3.4.1 yarl 1.9.4 zipp 3.18.1
I'm asking to copy the error you are seeing the leaderboard. Thanks
Hi Sorry for the delay. Please find below error message. I have also enclosed doc with screenshots. error screenshot.docx
STDERR Exception in thread ScriptRunner.scriptThread: Traceback (most recent call last): File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1045, in _bootstrap_inner self.run() File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "C:\Users\rajpr\AppData\Local\Programs\Python\Python311\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 267, in _run_script_thread ctx = ScriptRunContext( ^^^^^^^^^^^^^^^^^ TypeError: ScriptRunContext.init() missing 1 required positional argument: 'main_script_path'
Hi Team, Do you have any updates please
@rajpr5656 I'm unable to reproduce your error, however can you try:
tru.run_dashboard(force=True, port=8502)
@rajpr5656 please give this a try, thanks!
Closing due to inactivity
Bug Description Run 'tru.run_dashboard(port=8502)' Leaderboard shows error.
To Reproduce Not Sure on how to reproduce the steps as I am running standard Trulens Eval. I have enclosed screen shots as attachment and also all python packages. I am using python 3.12.2 version. Enclosed all Python packages below that are there. I have created a separate TruLens Leaderboard Issue.docx python environment and using there. However, I open Jupyter Lab in by generic folder. Package Version
aiohttp 3.9.3 aiosignal 1.3.1 alembic 1.13.1 altair 4.2.2 annotated-types 0.6.0 anyio 4.3.0 asgiref 3.8.1 asttokens 2.4.1 attrs 23.2.0 backoff 2.2.1 bcrypt 4.1.2 beautifulsoup4 4.12.3 blinker 1.7.0 bs4 0.0.2 build 1.2.1 cachetools 5.3.3 cargo 0.3 certifi 2024.2.2 charset-normalizer 3.3.2 chroma-hnswlib 0.7.3 chromadb 0.4.24 click 8.1.7 colorama 0.4.6 coloredlogs 15.0.1 comm 0.2.2 contourpy 1.2.0 cycler 0.12.1 dataclasses-json 0.6.4 debugpy 1.8.1 decorator 5.1.1 Deprecated 1.2.14 dill 0.3.8 dirtyjson 1.0.8 distro 1.9.0 entrypoints 0.4 executing 2.0.1 Faker 24.4.0 fastapi 0.110.0 favicon 0.7.0 filelock 3.13.3 flatbuffers 24.3.25 fonttools 4.50.0 frozendict 2.4.0 frozenlist 1.4.1 fsspec 2024.3.1 gitdb 4.0.11 GitPython 3.1.42 google-auth 2.29.0 googleapis-common-protos 1.63.0 greenlet 3.0.3 grpcio 1.62.1 h11 0.14.0 htbuilder 0.6.2 httpcore 1.0.5 httptools 0.6.1 httpx 0.27.0 huggingface-hub 0.22.1 humanfriendly 10.0 humanize 4.9.0 idna 3.6 importlib-metadata 6.11.0 importlib_resources 6.4.0 InstructorEmbedding 1.0.1 ipykernel 6.29.4 ipython 8.22.2 ipywidgets 8.1.2 jedi 0.19.1 Jinja2 3.1.3 joblib 1.3.2 jsonpatch 1.33 jsonpointer 2.4 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyter_client 8.6.1 jupyter_core 5.7.2 jupyterlab_widgets 3.0.10 kiwisolver 1.4.5 kubernetes 29.0.0 langchain 0.1.13 langchain-community 0.0.29 langchain-core 0.1.35 langchain-text-splitters 0.0.1 langsmith 0.1.36 llama-index 0.10.25 llama-index-agent-openai 0.2.1 llama-index-cli 0.1.11 llama-index-core 0.10.25.post1 llama-index-embeddings-huggingface 0.2.0 llama-index-embeddings-instructor 0.1.3 llama-index-embeddings-openai 0.1.7 llama-index-indices-managed-llama-cloud 0.1.5 llama-index-legacy 0.9.48 llama-index-llms-openai 0.1.13 llama-index-multi-modal-llms-openai 0.1.4 llama-index-program-openai 0.1.5 llama-index-question-gen-openai 0.1.3 llama-index-readers-file 0.1.12 llama-index-readers-llama-parse 0.1.4 llama-parse 0.4.0 llamaindex-py-client 0.1.15 lxml 5.1.0 Mako 1.3.2 Markdown 3.6 markdown-it-py 3.0.0 markdownlit 0.0.7 MarkupSafe 2.1.5 marshmallow 3.21.1 matplotlib 3.8.3 matplotlib-inline 0.1.6 mdurl 0.1.2 merkle-json 1.0.0 millify 0.1.1 minijinja 1.0.16 mmh3 4.1.0 monotonic 1.6 more-itertools 10.2.0 mpmath 1.3.0 multidict 6.0.5 munch 4.0.0 mypy-extensions 1.0.0 nest-asyncio 1.6.0 networkx 3.2.1 nltk 3.8.1 numpy 1.26.4 oauthlib 3.2.2 onnxruntime 1.17.1 openai 1.14.3 opentelemetry-api 1.23.0 opentelemetry-exporter-otlp-proto-common 1.23.0 opentelemetry-exporter-otlp-proto-grpc 1.23.0 opentelemetry-instrumentation 0.44b0 opentelemetry-instrumentation-asgi 0.44b0 opentelemetry-instrumentation-fastapi 0.44b0 opentelemetry-proto 1.23.0 opentelemetry-sdk 1.23.0 opentelemetry-semantic-conventions 0.44b0 opentelemetry-util-http 0.44b0 orjson 3.10.0 overrides 7.7.0 packaging 23.2 pandas 2.2.1 parso 0.8.3 pillow 10.2.0 pip 24.0 platformdirs 4.2.0 posthog 3.5.0 prometheus_client 0.20.0 prompt-toolkit 3.0.43 protobuf 4.25.3 psutil 5.9.8 pulsar-client 3.4.0 pure-eval 0.2.2 pyarrow 15.0.2 pyasn1 0.6.0 pyasn1_modules 0.4.0 pydantic 2.6.4 pydantic_core 2.16.3 pydeck 0.8.1b0 Pygments 2.17.2 pymdown-extensions 10.7.1 PyMuPDF 1.24.0 PyMuPDFb 1.24.0 pyparsing 3.1.2 pypdf 4.1.0 PyPika 0.48.9 pyproject_hooks 1.0.0 pyreadline3 3.4.1 python-dateutil 2.9.0.post0 python-decouple 3.8 python-dotenv 1.0.1 pytz 2024.1 pywin32 306 PyYAML 6.0.1 pyzmq 25.1.2 referencing 0.34.0 regex 2023.12.25 requests 2.31.0 requests-oauthlib 2.0.0 rich 13.7.1 rpds-py 0.18.0 rsa 4.9 safetensors 0.4.2 scikit-learn 1.4.1.post1 scipy 1.12.0 sentence-transformers 2.6.1 setuptools 69.2.0 six 1.16.0 smmap 5.0.1 sniffio 1.3.1 soupsieve 2.5 SQLAlchemy 2.0.29 st-annotated-text 4.0.1 stack-data 0.6.3 starlette 0.36.3 streamlit 1.32.2 streamlit-aggrid 1.0.0 streamlit-camera-input-live 0.2.0 streamlit-card 1.0.0 streamlit-embedcode 0.1.2 streamlit-extras 0.4.0 streamlit-faker 0.0.3 streamlit-image-coordinates 0.1.6 streamlit-keyup 0.2.3 streamlit-toggle-switch 1.0.2 streamlit-vertical-slider 2.5.5 striprtf 0.0.26 sympy 1.12 tenacity 8.2.3 threadpoolctl 3.4.0 tiktoken 0.6.0 tokenizers 0.15.2 toml 0.10.2 toolz 0.12.1 torch 2.2.2 tornado 6.4 tqdm 4.66.2 traitlets 5.14.2 transformers 4.39.1 trulens 0.13.4 trulens-eval 0.27.0 typer 0.11.0 typing_extensions 4.10.0 typing-inspect 0.9.0 tzdata 2024.1 urllib3 2.2.1 utils 1.0.2 uvicorn 0.29.0 watchdog 4.0.0 watchfiles 0.21.0 wcwidth 0.2.13 websocket-client 1.7.0 websockets 12.0 wheel 0.43.0 widgetsnbextension 4.0.10 wrapt 1.16.0 yarl 1.9.4 zipp 3.18.1
Expected behavior Expected to open Leaderboard and show results.
Relevant Logs/Tracebacks Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. If the issue is related to the TruLens dashboard, please also include a screenshot.
Environment:
Additional context If you need logs let me know where to find.
TruLens Leaderboard Issue.docx