444B / streamlit-analytics2

👀 Track & visualize user interactions with your streamlit app
MIT License
14 stars 2 forks source link

[FEATURE] tracking st.chat_input #44

Closed baswenneker closed 4 months ago

baswenneker commented 4 months ago

Awesome you're updating this Streamlit extension @444B 🙏

I'd like to track st.chat_input buttons for my AI chat app. Would that be possible?

Here's a screenshot: image

444B commented 4 months ago

@baswenneker thanks for raising the issue! I believe I just need to make a wrapper for it. I'll test it on a branch and if it works, I'll merge and release a new version today :) Update1 : Its harder than it looks! Need more time, working on it now

444B commented 4 months ago

Just a heads up that I am making progress on this and it can check how often the st.chat_inputshas been run but I am unsure whether it is respectful of the users privacy to return a list of the text inputs.

To handle such, we are looking at changing the way people configure the analytics with a settings screen For now, we can definitely find a way to share how many times chat_input has been run but we need a suitable way to handle user submitted data that respects user privacy

baswenneker commented 4 months ago

Ok, yeah I can imagine that tracking user input is a privacy risk. For me, I only care about how often a chat message was submitted. The contents are not important.

Btw, this is an analytics tool, it should only show usage stats, not the usage/input itself right?

444B commented 4 months ago

@baswenneker I am glad that we are on the same page regarding user privacy As for how often a chat message is submitted, this is feasible and I can get it done. For now, it will have to return the input as well since no PII is collected.

As to your question regarding inputs: I do believe that some input types are captured, such as st.input_text("Write your name") as seen below. image

Currently the analytics are not collecting by default any PII (Personally Identifiable Information) unless either the user submits it or the applicable developer chooses to collect it by asking the user to provide. In both cases it relies on the user to choose to submit info.

nit; I would actually define streamlit-analytics as a metrics tool rather than an analytics one, considering the above

On the roadmap we do have an upcoming integration with Google Analytics #16 and hopefully other analytic engines but this will be handled by this project in such a way that respects user content and awareness.

444B commented 4 months ago

@baswenneker This should now be in the latest version of streamlit_analytics2 (0.6.1) :)

The input text will be captured for now, until we can restructure the project to include settings for users to configure and since no PII is captured unless the user explicitly provides it

baswenneker commented 4 months ago

Great, I just tested it and it works. Thanks!

I do see something strange. It also captures "None" and I think it has something to do with me streaming the AI content back to the interface. It's not important but just something to keep in mind.

Here are two screenshots before and after chatting:

Before: image

After asking "What's the meaning of life?": image

I can't share my code (it's a closed source project for a client) but I think you can reproduce it with the example here: https://docs.streamlit.io/knowledge-base/tutorials/build-conversational-apps#build-a-simple-chatbot-gui-with-streaming

But note it is not really important because it only adds a "None" key to the chat_input analytics.

444B commented 4 months ago

Thanks for checking! I checked this now and am going to open an issue for it as a minor bug I was able to replicate that in my own environment