Support k-folds for Actions-based skills - Githubissues

cognitive-catalyst / WA-Testing-Tool

Scripts that run against Watson Assistant for K fold validation on training set, testing on blind test, and draw precision curves for comparison.

Apache License 2.0

78 stars 60 forks source link

Support k-folds for Actions-based skills #234

Open andrewrfreed opened 1 year ago

andrewrfreed commented 1 year ago

There are some APIs for working with skills in the v2 API - https://cloud.ibm.com/apidocs/assistant-v2#importskills - they are in a different API path than in Dialog skills.

Need to investigate these APIs, ideally the k-folds functionality can point to correct API for workspace (v1) or assistant (v2).

Update: There are APIs for Assistants too: https://cloud.ibm.com/apidocs/assistant-v2#createassistant

The old flow was:

Create k temporary skills
Train them
Test them
Delete them (leave no trace!)

The new flow can be:

Create k temporary assistants and skills (1-1 link)
Train them
Test them
Delete them (leave no trace!)

This should fit with the existing paradigm.