Revamp nlu testing - Githubissues

botpress / solutions

List of Botpress templates / examples for common problems. Kick start your chatbot development with these ready-to-go custom modules, actions, hooks, and tools!

https://botpress.com

56 stars 26 forks source link

Revamp nlu testing #19

Closed ierezell closed 1 year ago

ierezell commented 2 years ago

This PR aims to have a better tool for the NLU Testing tool.

It improves a few things:

[X] Added a UI
[X] Added possibility to give a bot.tgz to train
[X] Still have metrics and also details of each utterance in the test set.
[X] Pip installable package.

Todo :

[ ] Add a confusion matrix chart.
[ ] Add metrics for slots and entities
[ ] Few miscellaneous improvements (error handling / css )

How to use it :

Optional : Start a nlu server from botpress (go to botpress => yarn => yarn build => yarn start nlu) else you can connect to an existing nlu server.
pip install botpress_nlu_testing
`launch_bp_testing1

JustusNBB commented 2 years ago

Hi, I hope you don't mind me chiming in here and asking a few questions, I am interested in all kinds of testing approaches for bots:

why python (in contrast to most other bp tooling being typescript)?
what is the essence of this testing approach - and in contrast to e2e testing with e.g. newman or cypress?
what are the goals for the new GUI?
what about test automation? add Dockerfile with gui-disabled run?

ierezell commented 2 years ago

Hi @JustusNBB,

This is only a POC for now and is aimed to be used internally.

It's written in python as the data science team is using python most of the time and is simpler for others (marketing/sales) to jump in.
The testing approach is just to help people (internal but maybe external one day) to easily test their bots (note that it's a tool to be used outside of botpress querying only the nlu server and that it is not going to be a feature in the botpress/botpress repo).
- The GUI is extremely simple and was made so "non-technical" people can still use the tool.
- This is not made as an automation tool but just to help non-technical people on a repeating task (checking sanity of a bot).

Hope this help, Have a great day :)

ierezell commented 2 years ago

Note that the two new tools are also added as PyPI packages thanks to poetry