codeforequity-at
commented
4 years ago This is a valid concern, and I also experienced this myself.
Botium uploads a full dialogflow agent (model), including all intents and utterances, at once, and then it forces a training run. Unfortunately, we noticed that there is no way to know if the training actually finished, so we included a rather hacky process:
- after uploading the agent, there is a loop trying to do a basic intent resolution (just sending the literal text "hello")
- as soon as the dialogflow agent returns something meaningful (that is, not an error message), training is considered to be completed
When running my own tests, this was enough stable to do my evaluations, but maybe you have an idea how this can be solved better. I already considered introducing some fixed time slot to wait after uploading the model, to give Dialogflow time to finish the training, but then I didn't want to take a guess how long this could take.
shreshkhatri
According to Dialogflow, each time intent is updated, training phrase is added, entities are updated, the underlying algorithm gets updated. Even I have experienced that the Dialogflow model taking time to train when I add a training phrase.
I feel the same thing happens in case of testing a Dialogflow bot. When cross validation starts, training of phrases starts. I am thinking if the training for test purpose allows Dialogfolw some time to update its state of algorithm. I am confused, if training the model continuously and letting it to predict an utterance right after finishing training phase would classify that utterance correctly as the model might still be building its knowledge base. The task of classification may leads whereas algorithm building process may lag. This might produce incorrect results for precision, recall etc. I am wondering how much might I be correct with such kind of understanding. I hope to be cleared about this confusion. Thankyou