TruLens with Outside Logs example not working

[atulm-encora]

Bug Description I copied the example given at https://github.com/truera/trulens/blob/main/trulens_eval/examples/quickstart/existing_data_quickstart.ipynb. After running the example the dashboard is not showing any data for feedback. I ran this on Ubuntu 22 machine.

To Reproduce Copy code from above link and run it

Expected behavior It should show the scores for Answer relevance and Groundedness feedback functions.

Relevant Logs/Tracebacks Using legacy llama_index version None. Consider upgrading to 0.10.0 or later. ✅ In qs_relevance, input question will be set to record.main_input or Select.RecordInput . ✅ In qs_relevance, input statement will be set to record.app.retriever.get_context.rets[:] . ✅ In Groundedness, input source will be set to record.app.retriever.get_context.rets[:].collect() . ✅ In Groundedness, input statement will be set to record.main_output or Select.RecordOutput . ✅ In Answer Relevance, input prompt will be set to record.main_input or Select.RecordInput . ✅ In Answer Relevance, input response will be set to record.main_output or Select.RecordOutput . 🦑 Tru initialized with db url sqlite:///default.sqlite . 🛑 Secret keys may be written to the database. See the database_redact_keys option of Tru` to prevent this. Force stopping dashboard ... Starting dashboard ... Config file already exists. Skipping writing process. Credentials file already exists. Skipping writing process. Traceback (most recent call last): File "/path_to-dev_directory/test2.py", line 134, in tru.run_dashboard(force=True) File "/path_to-dev_directory/venv/lib/python3.11/site-packages/trulens_eval/tru.py", line 1096, in run_dashboard raise RuntimeError( RuntimeError: Dashboard failed to start in time. Please inspect dashboard logs for additional information.

You can now view your Streamlit app in your browser.

Dashboard started at http://ip-address:8501 . Network URL: http://ip-address:8501

External URL: http://ip-address:8501

Using legacy llama_index version None. Consider upgrading to 0.10.0 or later. Environment:

• OS: [Ubuntu]
• Python - 3.11
• TruLens 0.25.1
• Versions of other relevant installed libraries

Additional context Add any other context about the problem here.

[dosubot[bot]]

Based on the logs and traceback you provided, it seems like the TruLens dashboard failed to start in time. This error can be caused by several issues, including resource limitations, dependency issues, configuration errors, network problems, concurrent execution limits, or software bugs.

Given that you're using TruLens version 0.25.1, it's worth noting that there have been changes in the latest version, 0.26.0, that could potentially affect the dashboard functionality. These changes include enhancements in visualization capabilities and backend processing, which could directly impact the functionality, appearance, and performance of the dashboard. You can find more details about these changes in the Improvements section of the TruLens wiki.

As for the specific requirements and dependencies for running the TruLens dashboard on an Ubuntu 22 machine, you need to have conda installed and added to your path, Python 3.7, TensorFlow GPU version 1, Keras, and Matplotlib. You can either install TruLens via pip directly from PyPI using pip install trulens or for development purposes, clone the TruLens repository and install it locally with pip install -e . after navigating to the trulens_explain directory. These instructions are based on the content provided from the docs/trulens_explain/install.md file.

To troubleshoot the issue, you could try the following steps:

1. Check System Resources: Ensure that your system has sufficient resources (CPU, memory) to start the dashboard. If the system is under heavy load, try closing unnecessary applications or processes.

2. Update Dependencies: Make sure that all dependencies are correctly installed and up-to-date. You might want to consider creating a new virtual environment to avoid conflicts with other Python packages.

3. Check Configuration: Verify that the TruLens environment and the dashboard are correctly configured. This includes checking environment variables, API keys, and other settings required for the dashboard to operate.

4. Check Network: If the dashboard relies on external services or APIs, ensure that your system has a stable internet connection and can access these services.

5. Limit Concurrent Executions: If you're running multiple instances of the dashboard or other heavy processes concurrently, try limiting the number of concurrent executions.

6. Update TruLens: Consider updating to the latest version of TruLens (0.26.0) to benefit from the latest improvements and bug fixes.

If none of these steps resolve the issue, it would be helpful if you could provide more information about your system configuration and the exact steps you're following to start the dashboard. This could help identify any potential issues specific to your setup.

Sources

• Improvements
• docs/trulens_explain/install.md
• trulens_eval/examples/quickstart/py_script_quickstarts/llama_index_quickstart.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[atulm-encora]

Please note even after giving the error the dashboard starts. The real problem is its not showing any feedback scores. Looks like it failed to capture it hence there are no scores present.

I also updated the version to 0.26, but issue is not resolved.

[joshreini1]

@atulm-encora it can take some time for feedbacks to be computed. As they will get kicked off after the record is created, in some cases it can appear that the evaluations didn’t run when in fact you may just need to wait for them to finish generation. Can you confirm that they do appear after a longer amount of time?

[atulm-encora]

I kept the code running for 1 hour, but no records were generated even after 1 hour. The terminal kept showing following info and just continuing

[piotrm0]

Hi @atulm-encora , the screeshot you shared shows that 6 feedbacks were computed. You can ignore the last line that says "Waiting for Runs" as that will show "0/32" if there is nothing else to run. Does the dashboard show these 6 ?

[debthomaz]

Hello! I am having a similar issue, but in my case the feedback functions doesn't run at all, they all fail. Can you help me?

OS: Windows 10 Python - 3.10 TruLens - 0.28.2

[yuvneshtruera]

@DEBTHOMAZ Can you please create a new github issue and provide code/logs for us to reproduce the problem?

[debthomaz]

@yuvneshtruera Ok! Here it is https://github.com/truera/trulens/issues/1125

[yuvneshtruera]

@atulm-encora Please let us know if you are still facing this issue. I am closing this issue for now.