search

cognitive-catalyst / WA-Testing-Tool

Scripts that run against Watson Assistant for K fold validation on training set, testing on blind test, and draw precision curves for comparison.
Apache License 2.0
78 stars 61 forks source link

Support for new Actions assistants #215

Closed andrewrfreed closed 1 year ago

andrewrfreed commented 2 years ago

Need to update tool or confirm it works well for Action-based assistants, not just Dialog-based.

leighawilliamson commented 1 year ago

Many customers use this library of python functions inside notebooks that extract conversation logs and perform various analytics on the extracted log data. None of those customers are going to move to the New Watson Assistant until these functions work with the New WA. I can provide specific customer names if needed. This issue should have a high priority.

kevin2107 commented 1 year ago

I agree, we don't have actions in sight

andrewrfreed commented 1 year ago

@leighawilliamson @kevin2107 please note both capabilities (logs #227 and blind test #224 ) are addressed, see linked pull requests.

Thanks to many for review and help especially @lmazzoli who helped drive logs to completion.