truera / trulens

Evaluation and Tracking for LLM Experiments
https://www.trulens.org/
MIT License
1.84k stars 161 forks source link

TruLens Streamlit components #1224

Closed sfc-gh-jreini closed 1 day ago

sfc-gh-jreini commented 2 weeks ago

Items to add to release announcement:

https://github.com/truera/trulens/assets/171636836/3a5a0b53-27ed-442d-9139-6f118d332400

sfc-gh-dhuang commented 2 weeks ago

FYI you can open the PR as a Draft or still click Convert to draft near the reviewer section even after PR is opened :)

sfc-gh-jreini commented 5 days ago

One overall thing that confuses me is that is there a reason we're branding this stuff as trulens_st? Like is it important that it's streamlit? Why not like ui or something that way if we change it later it's not a misnomer or something.

Yeah, these are streamlit components targeted at addressing people already building with Streamlit. This is useful because:

review-notebook-app[bot] commented 2 days ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

sfc-gh-jreini commented 1 day ago

@sfc-gh-dkurokawa seems like your comments got squashed, but I think they should be addressed. I moved trulens_trace back to just calling tru - this should actually be okay because its a singleton.

sfc-gh-dkurokawa commented 1 day ago

@sfc-gh-dkurokawa seems like your comments got squashed, but I think they should be addressed. I moved trulens_trace back to just calling tru - this should actually be okay because its a singleton.

AFAICT, it's not technically a singleton, like if you give it a name you can have more then one. That is you can have:

tru_app1 = Tru(name="app1")
tru_app2 = Tru(name="app2)

as per the code it seems (I haven't tested it though).

sfc-gh-jreini commented 1 day ago

@sfc-gh-dkurokawa seems like your comments got squashed, but I think they should be addressed. I moved trulens_trace back to just calling tru - this should actually be okay because its a singleton.

AFAICT, it's not technically a singleton, like if you give it a name you can have more then one. That is you can have:

tru_app1 = Tru(name="app1")
tru_app2 = Tru(name="app2)

as per the code it seems (I haven't tested it though).

Ran the following test that seems to indicate it is a singleton/this would be okay. Let me know if you disagree

Set up two instances with different parameters and check fi they're equal:

from trulens_eval import Tru

tru_app1 = Tru(database_url='sqlite:///memory')
tru_app2 = Tru()
print(tru_app1 is tru_app2)
True

Manually check arguments from app 1.

args_tru_app1 = tru_app1.__dict__
print("Arguments used to initialize tru_app1:", args_tru_app1)
Arguments used to initialize tru_app1: {'db': SQLAlchemyDB(redact_keys=False, table_prefix='trulens_', engine_params={'url': 'sqlite:///memory',

Manually check arguments from app 2.

args_tru_app2 = tru_app2.__dict__
print("Arguments used to initialize tru_app2:", args_tru_app2)
Arguments used to initialize tru_app2: {'db': SQLAlchemyDB(redact_keys=False, table_prefix='trulens_', engine_params={'url': 'sqlite:///memory', ...

Arguments from app 2 are actually from app 1