Dialog Flow broken

[westlakem]

Dialog Flow does not delegate to code between filling slots.

VIRTUAL ALEXA -

    let resp = await alexa.utter('my intent')
    expect(resp.prompt).to.equal('What can we help you with?')
    resp = await alexa.utter('slot 1 answer')
    expect(resp.prompt).to.equal('Which part?')
    resp = await alexa.utter('1');

SERVER OUTPUT - VIRTUAL ALEXA

The log shows 2 inputs, one where dialogState == null and one where dialogState == 'COMPLETED' without any state in between:

2018-12-10 15:08:33 DEBUG class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-external.request.8e703029-9479-4cb4-90fa-8c1a400afa45
        timestamp: 2018-12-10T20:08:33Z
        locale: en-US
    }
    dialogState: null
    intent: class Intent {
        name: Intent1
        slots: {slot1=class Slot {
            name: slot1
            value: null
            confirmationStatus: NONE
            resolutions: null
        }, slot2=class Slot {
            name: slot2
            value: null
            confirmationStatus: NONE
            resolutions: null
        }}
        confirmationStatus: null
    }
}

Optional[class Response {
    outputSpeech: null
    card: null
    reprompt: null
    directives: [class DelegateDirective {
        class Directive {
            type: Dialog.Delegate
        }
        updatedIntent: class Intent {
            name: Intent1
            slots: {slot1=class Slot {
                name: slot1
                value: null
                confirmationStatus: NONE
                resolutions: null
            }, slot2=class Slot {
                name: slot2
                value: null
                confirmationStatus: NONE
                resolutions: null
            }}
            confirmationStatus: null
        }
    }]
    shouldEndSession: null
    canFulfillIntent: null
}]

2018-12-10 15:08:33 DEBUG - class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-external.request.3720b4bf-225a-45d0-908a-f340bffc503d
        timestamp: 2018-12-10T20:08:33Z
        locale: en-US
    }
    dialogState: COMPLETED
    intent: class Intent {
        name: Intent1
        slots: {slot1=class Slot {
            name: slot1
            value: slot 1 answer
            confirmationStatus: NONE
            resolutions: class Resolutions {
                resolutionsPerAuthority: [class Resolution {
                    authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                    status: class Status {
                        code: ER_SUCCESS_MATCH
                    }
                    values: [class ValueWrapper {
                        value: class Value {
                            name: Stuck
                            id: null
                        }
                    }]
                }]
            }
        }, slot2=class Slot {
            name: slot2
            value: 1
            confirmationStatus: NONE
            resolutions: null
        }}
        confirmationStatus: NONE
    }
}
Optional[class Response {
    outputSpeech: class SsmlOutputSpeech {
        class OutputSpeech {
            type: SSML
        }
        ssml: <speak>Completed</speak>
    }
    card: null
    reprompt: null
    directives: null
    shouldEndSession: null
    canFulfillIntent: null
}]

When I test this through the alexa developer portal, I get multiple states in between and the dialog starting state is STARTED not null:

2018-12-10 19:53:06 DEBUG - class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-api.request.555d79fa-6add-4683-a977-f07897e022c9
        timestamp: 2018-12-10T19:53:05Z
        locale: en-US
    }
    dialogState: STARTED
    intent: class Intent {
        name: Intent1
        slots: {slot2=class Slot {
            name: slot2
            value: null
            confirmationStatus: NONE
            resolutions: null
        }, slot1=class Slot {
            name: slot1
            value: null
            confirmationStatus: NONE
            resolutions: null
        }}
        confirmationStatus: NONE
    }
}
Optional[class Response {
    outputSpeech: null
    card: null
    reprompt: null
    directives: [class DelegateDirective {
        class Directive {
            type: Dialog.Delegate
        }
        updatedIntent: class Intent {
            name: Intent1
            slots: {slot2=class Slot {
                name: slot2
                value: null
                confirmationStatus: NONE
                resolutions: null
            }, slot1=class Slot {
                name: slot1
                value: null
                confirmationStatus: NONE
                resolutions: null
            }}
            confirmationStatus: NONE
        }
    }]
    shouldEndSession: null
    canFulfillIntent: null
}]

2018-12-10 19:53:36 DEBUG - class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-api.request.6bbc3b8d-8931-46b1-a78b-87c49e16b82d
        timestamp: 2018-12-10T19:53:35Z
        locale: en-US
    }
    dialogState: IN_PROGRESS
    intent: class Intent {
        name: Intent1
        slots: {slot2=class Slot {
            name: slot2
            value: null
            confirmationStatus: NONE
            resolutions: null
        }, slot1=class Slot {
            name: slot1
            value: slot 1 answer
            confirmationStatus: NONE
            resolutions: class Resolutions {
                resolutionsPerAuthority: [class Resolution {
                    authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                    status: class Status {
                        code: ER_SUCCESS_MATCH
                    }
                    values: [class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 1
                            id: d418720232c238736820b1e376d3e70b
                        }
                    }, class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 2
                            id: 0d9c296293117e8ef114e26d5c3720a8
                        }
                    }]
                }]
            }
        }}
        confirmationStatus: NONE
    }
}
Optional[class Response {
    outputSpeech: null
    card: null
    reprompt: null
    directives: [class DelegateDirective {
        class Directive {
            type: Dialog.Delegate
        }
        updatedIntent: class Intent {
            name: Intent1
            slots: {slot2=class Slot {
                name: slot2
                value: null
                confirmationStatus: NONE
                resolutions: null
            }, slot1=class Slot {
                name: slot1
                value: slot 1 answer
                confirmationStatus: NONE
                resolutions: class Resolutions {
                    resolutionsPerAuthority: [class Resolution {
                        authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                        status: class Status {
                            code: ER_SUCCESS_MATCH
                        }
                        values: [class ValueWrapper {
                            value: class Value {
                                name: slot 1 answer 1
                                id: d418720232c238736820b1e376d3e70b
                            }
                        }, class ValueWrapper {
                            value: class Value {
                                name: slot 1 answer 2
                                id: 0d9c296293117e8ef114e26d5c3720a8
                            }
                        }]
                    }]
                }
            }}
            confirmationStatus: NONE
        }
    }]
    shouldEndSession: null
    canFulfillIntent: null
}]

2018-12-10 19:53:42 DEBUG - class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-api.request.2ee425bc-6aa4-4586-ae3a-4512359f452c
        timestamp: 2018-12-10T19:53:42Z
        locale: en-US
    }
    dialogState: IN_PROGRESS
    intent: class Intent {
        name: Intent1
        slots: {slot2=class Slot {
            name: slot2
            value: 1
            confirmationStatus: NONE
            resolutions: null
        }, slot1=class Slot {
            name: slot1
            value: slot 1 answer
            confirmationStatus: NONE
            resolutions: class Resolutions {
                resolutionsPerAuthority: [class Resolution {
                    authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                    status: class Status {
                        code: ER_SUCCESS_MATCH
                    }
                    values: [class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 1
                            id: d418720232c238736820b1e376d3e70b
                        }
                    }, class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 2
                            id: 0d9c296293117e8ef114e26d5c3720a8
                        }
                    }]
                }]
            }
        }}
        confirmationStatus: NONE
    }
}

Optional[class Response {
    outputSpeech: null
    card: null
    reprompt: null
    directives: [class DelegateDirective {
        class Directive {
            type: Dialog.Delegate
        }
        updatedIntent: class Intent {
            name: Intent1
            slots: {slot2=class Slot {
                name: slot2
                value: 1
                confirmationStatus: NONE
                resolutions: null
            }, slot1=class Slot {
                name: slot1
                value: slot 1 answer
                confirmationStatus: NONE
                resolutions: class Resolutions {
                    resolutionsPerAuthority: [class Resolution {
                        authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                        status: class Status {
                            code: ER_SUCCESS_MATCH
                        }
                        values: [class ValueWrapper {
                            value: class Value {
                                name: slot 1 answer 1
                                id: d418720232c238736820b1e376d3e70b
                            }
                        }, class ValueWrapper {
                            value: class Value {
                                name: slot 1 answer 2
                                id: 0d9c296293117e8ef114e26d5c3720a8
                            }
                        }]
                    }]
                }
            }}
            confirmationStatus: NONE
        }
    }]
    shouldEndSession: null
    canFulfillIntent: null
}]

2018-12-10 19:53:42 DEBUG - class IntentRequest {
    class Request {
        type: IntentRequest
        requestId: amzn1.echo-api.request.00628a6b-82a8-466c-ac19-975ad6c81130
        timestamp: 2018-12-10T19:53:42Z
        locale: en-US
    }
    dialogState: COMPLETED
    intent: class Intent {
        name: Intent1
        slots: {slot1=class Slot {
            name: slot1
            value: slot 1 answer
            confirmationStatus: NONE
            resolutions: class Resolutions {
                resolutionsPerAuthority: [class Resolution {
                    authority: amzn1.er-authority.echo-sdk.amzn1.ask.skill.1d4e6648-fdad-457f-831a-6124d5d8f032.slot1s
                    status: class Status {
                        code: ER_SUCCESS_MATCH
                    }
                    values: [class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 1
                            id: d418720232c238736820b1e376d3e70b
                        }
                    }, class ValueWrapper {
                        value: class Value {
                            name: slot 1 answer 2
                            id: 0d9c296293117e8ef114e26d5c3720a8
                        }
                    }]
                }]
            }
        }, slot2=class Slot {
            name: slot2
            value: 1
            confirmationStatus: NONE
            resolutions: null
        }}
        confirmationStatus: NONE
    }
}
Optional[class Response {
    outputSpeech: class SsmlOutputSpeech {
        class OutputSpeech {
            type: SSML
        }
        ssml: <speak>Completed</speak>
    }
    card: null
    reprompt: null
    directives: null
    shouldEndSession: null
    canFulfillIntent: null
}]

[jkelvie]

Hi Michael, I apologize for the slow response. This is a tricky issue to fix, and I believe represents a significant oversight on our part regarding the dialog behavior.

I do not have an exact time yet for a fix, but in the meantime, have you looked at our end-to-end testing? One advantage of it is that it interacts with the real Alexa. You can see an example of a dialog testing scenario here: https://github.com/alexa/skill-sample-nodejs-college-finder/blob/master/test/e2e/FullSearchHandler.e2e.yml#L26

Our documentation on end-to-end testing is here: https://read.bespoken.io/end-to-end/getting-started

The end-to-end testing does require the skill and interaction model be deployed to test, so it is not without tradeoffs. But if you want to tests a delegated dialog in which you are "taking over" the dialog, it is the best solution we can offer at the moment.

Separately, we will update this ticket once we get this issue scheduled and have a time-frame in mind for a fix.

[dgreene1]

Looking forward to this fix @jkelvie. As always, thank you for the great work your team is doing! :)

[westlakem]

Any updates on this? @dgreene1 @jkelvie

[jkelvie]

We have not had time to look at this. I recommend using the end-to-end testing, as mentioned above. Alternatively, you can use our new SMAPI Simulation support - that will also be useful for testing skills using the Dialog interface. More information here: https://github.com/bespoken-samples/GuessThePrice/tree/master/test/simulation

[jkelvie]

Hi @westlakem , our latest version 0.7.1 addresses this issue.

We have scaled back the scope of what our emulator does for dialog management - instead of seeking to emulate the actual interactions from a user's perspective, it focuses on creating the correct payloads that will go back and forth to your skill.

For an example of how to use it, look here: https://github.com/bespoken-samples/skill-sample-nodejs-petmatch/blob/master/lambda/custom/index-test.js

Let me know if you have any questions, and good luck with your testing.