SMAPI's Simulation API Failed with An unexpected error occured

[chandora]

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report  
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

Expected Behavior

I invoked a skill via the SMAPI Simulation API on the following conditions:

• The skill has an intent with a slot.
• The auto delegation is enabled for the slot.
• The input text to the Simulation API has a phrase to match the intent without fulfilling the slot.

I expected to receive a response with 'SUCCESSFUL' status and asking to fulfill the slot.

Current Behavior

The API returns the 'FAILED' status and "An unexpected error occurred." message in API response (response.result.error) as follows:

{ "message": "An unexpected error occurred." }

When I disabled the auto delegation for the slot and provided the following handler in the skill, the Simulation API returned a response with the 'SUCCESSFUL' status. I could fulfill the slot by invoking the API with a new phrase containing the slot value.

const DialogDelegationHandler = {
    canHandle(handlerInput) {
        const envelope = handlerInput.requestEnvelope;
        if (Alexa.getRequestType(envelope) === 'IntentRequest') {
            const dialogState = Alexa.getDialogState(envelope);
            if (dialogState === 'STARTED' || dialogState === 'IN_PROGRESS') {
                return true;
            }
        }
        return false;
    },
    handle(handlerInput) {
        const request = Alexa.getRequest(handlerInput.requestEnvelope);
        return handlerInput.responseBuilder
            .speak('')
            .addDelegateDirective(request.intent)
            .getResponse();
    }
};

Possible Solution

Steps to Reproduce (for bugs)

The following test code was used to create the issue. "drink stand" is the name of skill.

import * as Smapi from 'ask-smapi-sdk';
import * as SmModel from 'ask-smapi-model';
import * as config from './config.json';

type SmapiClient = SmModel.services.skillManagement.SkillManagementServiceClient;
type SmApiResponse = SmModel.v2.skill.simulations.SimulationsApiResponse;
type SmApiRequest = SmModel.v2.skill.simulations.SimulationsApiRequest;

const refreshTokenConfig: Smapi.RefreshTokenConfig = {
    clientId: config.client_id,
    clientSecret: config.client_secret,
    refreshToken: config.refresh_token
};

const smapiClient = new Smapi.StandardSmapiClientBuilder()
    .withRefreshTokenConfig(refreshTokenConfig)
    .client();

function sleep(ms: number) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

const simulationRequest: SmApiRequest = {
    session: {
        mode: 'FORCE_NEW_SESSION'
    },
    input: {
        content: 'ask drink stand something drink'
    },
    device: {
        locale: 'en-US'
    }
}

async function doTest(smapiClient: SmapiClient, skillId: string) {
    let apiResponse: SmApiResponse;
    apiResponse = await smapiClient.simulateSkillV2(skillId, 'development', simulationRequest);

    const simulationId = apiResponse.id;

    if (simulationId) {
        process.stdout.write('Wating Alexa response .');
        while (apiResponse.status === 'IN_PROGRESS') {
            process.stdout.write('.');
            await sleep(1000);
            apiResponse = await smapiClient.getSkillSimulationV2(skillId, 'development', simulationId);
        }
        console.log(' completed');
    }

    if (apiResponse.status === 'FAILED') {
        console.log(JSON.stringify(apiResponse.result?.error, null, 2));
    }
    else {
        console.log(JSON.stringify(apiResponse.result, null, 2));
    }
}

const skillId = config.skill_id_En;

(async () => await doTest(smapiClient, skillId))();

Context

Your Environment

SMAPI SDK for Nodejs

• ask-smapi-sdk 1.3.0
• ask-smapi-model 1.15.2

Node.js and NPM Info

• Node.js version used for development: 12.13.1
• NPM version used for development: 7.10.0

[ShenChen93]

Hi @chandora ,

Thanks for using our SDK ! Could you please provide your skill ID ? Thus we could check the logs on service side

Thanks, Shen

[chandora]

@ShenChen-Amazon

Thank you for checking this. The skill ID is as follows:

amzn1.ask.skill.322a16f8-88ee-4e53-8e07-feec69668649

The last run was the error case.

Thank you, chandora

[ShenChen93]

Hi @chandora ,

Confirmed with the service team. It turns out this is the expected behavior. With auto delegation Alexa will prompt for slot elicitation without invoking the skill.

I expected to receive a response with 'SUCCESSFUL' status and asking to fulfill the slot.

this does not happen, the service side do not return any responses where the skill was not involved, and the simulation api fail because of missing skill I/O.

Thanks, Shen

[chandora]

Hi @ShenChen-Amazon ,

Thank you for the quick response.

Is there any way to test a conversation with the auto delegation? I know that the Alexa Simulator in the Developer Console supports it. So I believe there might be an API to do it.

If no API is available for it, I will consider submitting a feature request.

I am developing an automated skill test driver, and need this capability for integration tests involving the auto delegation as a part of conversations.

By the way, I would suggest updating the message of the API. "Unexpected error occurred" is "expected behavior" sounds no good :-)

Thank you, chandora

[ShenChen93]

Hi @chandora ,

Unfortunately, the auto delegation is managed by the Dialog Management service, currently the simulate api doesn't support simulate DM. This is a very good point for the simulate api. I've raised a feature request on the service team.

You could post this feature request at Alexa forums as well and vote for it, thus it may be prioritized by the service team if lot of people has the same request.

Thanks, Shen

[chandora]

Hi @ShenChen-Amazon ,

It's sad news but I understood. I will post my feature request once I release the test driver.

Thank you for your support!

Regards, chandora