RasaHQ / rasalit

Visualizations and helpers to improve and debug machine learning models for Rasa Open Source
Apache License 2.0
306 stars 62 forks source link

Adding labeled utterances for reference in Bulk Labeling Tool #54

Closed oppasource closed 3 years ago

oppasource commented 3 years ago

Thanks for the amazing work!! Loved it.

Bulk Labeling Tool currently only takes unlabeled utterances. Is it possible to visualize few already labeled utterances as well (possibly with different color) in the same semantic space. This will allow users to visualize separation of clusters more clearly. For example there might be confusion of considering a sub-cluster or a bigger super-cluster while labeling. Colored dots(few labeled samples) can help guide the manual cluster selection process.

Also a drop down menu for all available intent that are already present in the current labeled version would go a long way I feel.

sara-tagger commented 3 years ago

Thanks for raising this issue, @yennycheung will get back to you about it soon✨

Please also check out the docs and the forum in case your issue was raised there too 🤗
koaning commented 3 years ago

So, technically, we already support labelled utterances but you need to pass extra settings to the human-learn code. As you can see in the docs here, it's totally allowed. You probably need to add something like below to the InteractiveCharts call.

clf = InteractiveCharts(df, labels="whatever-label-name")
koaning commented 3 years ago

A drop-down menu might work well here, but I should start by admitting that there is a reason why the current state of the project is a Jupyter notebook as opposed to a python package; it is still very much work in progress. One thing that I'm also exploring is if it makes sense to add a model in the mix for active learning. By releasing it as a notebook my hope is that people feel free to hack around themselves as well and that they might let me know what works well.

oppasource commented 3 years ago

Thanks taking time out for your helpful response.