search

cognitive-catalyst / WA-Testing-Tool

Scripts that run against Watson Assistant for K fold validation on training set, testing on blind test, and draw precision curves for comparison.
Apache License 2.0
78 stars 60 forks source link

Enhancement: Support Directly referencing an @Entity as an intent example #32

Open Stephen-Broadhurst opened 5 years ago

Stephen-Broadhurst commented 5 years ago

Firstly - great tool thank you. The WA classifier now supports using entities within intent training. For instance "Can I get a @PhoneModelName? " https://console.bluemix.net/docs/services/conversation/intents.html#defining-intents

This is very useful when reusing bot training across multiple bots for instance "Hello @me" where the intent training can be ported across and @me @otherpeople be controlled as entities.

At the moment when creating test data, the WA-Testing-Tool, submits "Hello @me" without replacing the @me with a literal from the entity training. This means that these values almost always fail, even though they would pass if @me replaced with one of the literal values.

Ask is to download the entity training with the intent training. When the intent training trains the workspace, @entity should be passed as currently. But when the test set is created, the @entity should be replaced with a random synonym/literal from the entity training set.

amblock commented 5 years ago

Hi Stephen - We had been struggling to come up with a solution for how to integrate entity references into the tool. I'm a fan of the idea you proposed.