Library Alpha Discussion: Provide action package and get coverage results (i.e. % of intents covered in your tests)

[atulep]

Action package lists all of the intents in your project. We can potentially utilize this information to calculate coverage of intents invoked in your tests. For example, suppose your Action will contain 2 intents: A and B. If your test will look like,

action.startTest(‘testing happy path’, action => {
  return action.startConversation().
              then((res) => { action.send(/* some phrase to trigger intent A */) })
})

, then coverage output will be “50 % intents covered (½)”.

[yoichiro]

@atulep Is there a way to get a list of intents dynamically from Actions on Google? Also, how can we recognize that the intent A was triggered in the example above?

[Fleker]

I don't think there's a way to dynamically get intents, but if you pass in the action package from Dialogflow or Actions SDK, we should be able to get all intents. In the action response, you can see the intent that matches. We can track all matched intents versus all intents in general.

[yoichiro]

I tested some behaviors, and I noticed the following:

• In the case of using Dialogflow, I can receive the responseMetadata property value in the response of the Assistant. Also, I can receive the possibleIntents which is the list of available intents at the same time. Each value is like UUID and refers each intent defined in the Dialogflow agent.
• In the case of using Actions SDK, the responseMetadata property value is not included in the response of the Assistant. And, although I can receive the possibleIntents, it always includes the assistant.intent.action.TEXT only.

In the case of using Dialogflow, almost intents may be collected from the possibleIntents values by progressing tests. If developers can prepare enough count of test cases, by storing the intent's IDs from the possibleIntents, I guess that we can create the population of the intents.

[Fleker]

If we have a config, where you pass in your Dialogflow zip, we can grab all the ids up front by reading in the files in intents/ (that don't end with says_*.json) to obtain the id

[yoichiro]

@Fleker Yes, I think that it is a simple and certain idea. Basically, this lib's users write a test code. Therefore, they can set the file path of the Dialogflow exported zip file on the test code. For example, my idea is like below:

const { ActionsOnGoogleAva } = require("actions-on-google-testing");
const action = new ActionsOnGoogleAva(require("./credentials.json"));

action.turnOnCoverage({
  dialogflowZipFilePath: '...'
});

action.startTest('...', action => { ... });
action.startTest('...', action => { ... });
...

When starting tests, that is, when the test.before(...) callback function called, we can load the zip file and retrieve the intents ID list. Then, when the test.after(...) callback function called, we can output the coverage report. These processes can be written in the ActionsOnGoogleAva class, I guess.

How do you think above?

[Fleker]

Yeah that's primarily what I envision. Would it make more sense to incorporate it into the constructor?

const action = new ActionsOnGoogleAva(require("./credentials.json", {
    actionPackage: '/path/to/dialogflow.zip'
}));

[yoichiro]

Yes, the idea which uses the constructor also looks good to me. I intend to start implementing this feature.

@Fleker Nick, I want to confirm one point. Is the name actionPackage ok? Actually, the exported zip file from Dialogflow is not an Action Package, is a Dialogflow Agent setting. Is it not possible that developers misunderstand it as a different thing?

[Fleker]

You're correct, technically it should accept either an actionPackage or dialogflowZip parameter.

[maaaaaaaax]

Due to more than 3 months of inactivity, this issue is being closed.

If you think the resolution is in error, please re-open the request with fresh information.