bug(audio-player): attributes not preserved when audio player is playing

[CoreyCole]

We have an intent, LogoutIntent, when we call it, it prompts the user to confirm:

Are you sure you want to logout?

So it emits type :ask to the user, expecting a followup confirmation response from them (yes/no). In this followup confirmation yes/no intent, we check the attributes to see which intent is being confirmed by the user. When the audio player is not playing, the attributes are preserved and our followup confirmation intent works as expected. But, when the audio player is playing, the attributes object is empty even though the session with the user has not ended.

When testing with a real Alexa device, it works both when the audio player is playing and not.

Here is the test we are having trouble with:

test('LogoutIntent should log out', async () => {
    const deviceId = uuid();
    alexa.context().device().setID(deviceId);
    expect(alexa.context().device().id()).toEqual(deviceId);

    const logoutResponse = await alexa.intend('LogoutIntent');
    expect(logoutResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutResponse['response'].outputSpeech.ssml).toMatchSnapshot();
    // attributes are defined to guide the confirmation intent

    const logoutConfirmationResponse = await alexa.utter('yes');
    // attributes are defined here, works as intended
    expect(logoutConfirmationResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutConfirmationResponse['response'].outputSpeech.ssml).toMatchSnapshot();

    const launchResponse = await alexa.launch();
    expect(launchResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(launchResponse['response'].outputSpeech.ssml).toMatchSnapshot();

    const loginResponse = await alexa.utter(`My name is ${testConstants.userName}`);
    expect(loginResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(loginResponse['response'].outputSpeech.ssml).toMatchSnapshot();

    const usernameConfirmationResponse = await alexa.utter('yes');
    expect(usernameConfirmationResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(usernameConfirmationResponse['response'].outputSpeech.ssml).toMatchSnapshot();

    const logoutResponse2 = await alexa.intend('LogoutIntent');
    expect(logoutResponse2['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutResponse2['response'].outputSpeech.ssml).toMatchSnapshot();

    const logoutConfirmationResponse1 = await alexa.utter('yes');
    // attributes are defined here, works as intended
    expect(logoutConfirmationResponse1['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutConfirmationResponse1['response'].outputSpeech.ssml).toMatchSnapshot();

    const loginResponse1 = await alexa.utter(`My name is ${testConstants.userName}`);
    expect(loginResponse1['response'].outputSpeech.ssml).toBeDefined();
    expect(loginResponse1['response'].outputSpeech.ssml).toMatchSnapshot();

    const usernameConfirmationResponse1 = await alexa.utter('yes');
    expect(usernameConfirmationResponse1['response'].outputSpeech.ssml).toBeDefined();
    expect(usernameConfirmationResponse1['response'].outputSpeech.ssml).toMatchSnapshot();

    const joinIntentResponse = await alexa.intend('JoinIntent', { channelname: testConstants.channelName });
    expect(joinIntentResponse['response'].outputSpeech.ssml).toBeDefined();
    expect(joinIntentResponse['response'].outputSpeech.ssml).toMatchSnapshot();

    const channelConfirmationResponse = await alexa.utter('yes');
    // at this point, the audio player starts
    expect(channelConfirmationResponse['response'].outputSpeech).toBeUndefined();
    expect(channelConfirmationResponse['response'].shouldEndSession).toBeDefined();
    expect(channelConfirmationResponse['response'].shouldEndSession).toBeTruthy();

    const logoutResponse3 = await alexa.intend('LogoutIntent');
    expect(logoutResponse3['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutResponse3['response'].outputSpeech.ssml).toMatchSnapshot();

    // when testing with alexa device, the attributes are defined here as expected
    // but when testing with virtual-alexa, the attributes object is empty (defined, but empty {})
    // (even though the session is still open i.e. emit ":ask")
    const logoutConfirmationResponse2 = await alexa.intend('SoloSlotIntent', { solo_slot_value: 'yes' });
    // we have tried this with both "utter" and "intend", neither works as expected
    expect(logoutConfirmationResponse2['response'].outputSpeech.ssml).toBeDefined();
    expect(logoutConfirmationResponse2['response'].outputSpeech.ssml).toMatchSnapshot();
}, 15000);

[jkelvie]

So, to be clear, there will not be session attributes while the audio player is playing. Is it that new session attributes defined after it stops are not preserved?

[CoreyCole]

In our experience on actual alexa devices, when the audio player is playing and you wake alexa, the audio player stops (PlaybackStopped) until the user's session with alexa is complete (the blue light goes off). In the case of a :tell emit(), the audio player starts back up (PlaybackStarted) right after alexa prompts the response to the user.

In the case of an :ask emit(), the audio player still does not start back up until the session with the user is ended (when a :tell is ultimately emitted). On real alexa devices, the attributes object is preserved between the :ask response and the following intent (even when the audio player is waiting to start back up).

So, the audio player isn't actually "playing". It is stopped waiting for the session with the user to complete. But, we believe it has something to do with the audio player because the attributes are preserved correctly in our tests before the audio player has started and it is an empty object following an :ask emit, after the audio player has started, but is currently stopped waiting for the session with the user to end.

So to answer your question, yes the attributes is an empty object while the audio player is stopped. More specifically, after and :ask has been emitted to the user while the audio player is temporarily stopped while it waits for the session with the user to complete.

[CoreyCole]

Sorry, I closed on accident. I thought I found the problem on our end, but we're still experiencing the same behavior.

[jkelvie]

Hi @CoreyCole - I'm having difficulty following this. Can you state the issue in terms of: Expected Behavior - what you wanted to happen Actual Behavior - what actually happened Reproduction Steps - the exact steps needed (ideally no more and no less) to reproduce the issue (ideally also with the code involved)

Being as concise as possible as well as including code makes it more likely we can address the issue.

Thank you, John

[CoreyCole]

Expected Behavior

Alexa attributes are always preserved when emitting type :ask and getting a followup response from the user. Just like with real alexa devices.

Actual Behavior

After the audio player has started playing (in virtual-alexa), Alexa attributes object is empty following an emit of type :ask when getting a followup response from the user.

Reproduction Setup

Create a minimal skill with

• an audio player
• an intent to ask a question
• an intent to capture a response to that question

Inside the intent to ask a question, set an Alexa attribute, i.e.

askQuestion(intentHandler: Alexa.Handler<Alexa.Request>) {
  const prompt = 'what is your name?';
  intentHandler.attributes.lastSpeech = prompt;
  // intentHandler.response.speak(prompt).listen(reprompt); alternative syntax
  intentHandler.emit(':ask', prompt, prompt);
}

Inside the intent to capture a response to the question, simply print the attributes object.

followUp(intentHandler: Alexa.Handler<Alexa.Request>) {
  // this will have `{ lastSpeech: "what is your name" }` before the audio player has started
  // this will be an empty object after the audio player has started
  console.log(intentHandler.attributes); 
}

Reproduction Steps

Create a virtual-alexa test

test('Attributes should be preserved', async () => {
  const askRes = await alexa.intend('AskQuestionIntent');
  const followupRes = await alexa.intend('FollowUpIntent');
  // you should see lastSpeech printed in the attributes, as expected

  const playRes = await alexa.indend('StartAudioIntent');
  const askRes2 = await alexa.intend('AskQuestionIntent');
  const followupRes2 = await alexa.intend('FollowUpIntent');
  // you should see an empty attributes object printed
}, 15000);

You can also test the skill using a real alexa device. You will see that the attributes object is defined and lastSpeech is in there even after the audio has started.

[CoreyCole]

@jkelvie We are still experiencing problems with the session attributes not being preserved while the audio player is playing. If it is still unclear what I'm talking about, could you let me know? You can see the session attributes in the alexa test simulator here:

[CoreyCole]

Is there any way we could make a feature to manually set the session.attributes object in our tests before running the UserIntent? i.e.

test('Attributes should be preserved', async () => {
  const askRes = await alexa.intend('AskQuestionIntent');
  const followupRes = await alexa.intend('FollowUpIntent');
  // you should see lastSpeech printed in the attributes, as expected

  const playRes = await alexa.indend('StartAudioIntent');
  const askRes2 = await alexa.intend('AskQuestionIntent');

  // manually set the attributes before following up
  alexa.session().attributes = { "lastSpeech": "what we expect to see" };
  const followupRes2 = await alexa.intend('FollowUpIntent');
  // you should not not see an empty attributes object printed because we manually set it in our test
}, 15000);

I could also see it being an optional parameter to the intend() function:

test('Attributes should be preserved', async () => {
  const askRes = await alexa.intend('AskQuestionIntent');
  const followupRes = await alexa.intend('FollowUpIntent');
  // you should see lastSpeech printed in the attributes, as expected

  const playRes = await alexa.indend('StartAudioIntent');
  const askRes2 = await alexa.intend('AskQuestionIntent');

  // manually set the attributes before following up (empty slots object)
  const followupRes2 = await alexa.intend('FollowUpIntent', {}, { "lastSpeech": "what we expect to see"});
  // you should not not see an empty attributes object printed because we manually set it in our test
}, 15000);

This is a huge problem for us because our intents rely on session attributes to function, and these are currently untestable with virtual-alexa once the audio player has started playing. Ideally the bug is fixed and session attributes are actually preserved while the audio player is playing, but this feature could be a workaround if it is more simplistic to implement--I have no idea--let me know 😄

[jkelvie]

@CoreyCole - I am not sure how to think about this issue, because by definition, the session ends when the audio player starts and the attributes should go away. I don't really understand what followup response you are expecting once the AudioPlayer does start.

That said, have you looked at our filter function? It allows you to manipulate the request before it is sent to your skill: https://github.com/bespoken/virtual-alexa#using-the-request-filter

We do use it all the time to handle test scenarios that are not readily handled by Virtual Alexa.

[CoreyCole]

Sorry, I'm not being clear enough. Yes, I know the session ends and session data is cleared once the audio player starts.

But, I'm talking about the user interrupting the audio player with an intent--particularly and intent that keeps the session open.

When a user invokes an intent while the audio player is playing, the audio player stops while alexa handles that intent. In the case of an :ask response, the audio player stays interrupted until that interrupting-session ends. Throughout this momentary pause of the audio player, the session attributes are preserved on real alexa devices, but not in virtual alexa.

[jkelvie]

Okay, I think I understand now - I would recommend using the filter for that scenario - we will keep this issue open, and look to get it fixed, but I don't have an ETA for that at the moment

[CoreyCole]

Sounds good, I will let you know here if using the filter solves the problem for us.

[CoreyCole]

Yes, setting the attributes manually using the request filter worked for us! Thank you for pointing that out. If you give me a spec for the fix I'd be happy to work on it and submit a PR - it is hacktoberfest after all 😄

[jkelvie]

I spent some time looking into this - I still do not have a definitive answer, but my best guess is that the issue relates to this line of code: https://github.com/bespoken/virtual-alexa/blob/master/src/impl/SkillInteractor.ts#L174

When the audio player is paused, we do not account for an extended interaction with the user. Instead, we expect a response that ends the session, and then for the audio player to immediately resume.

Adding a check on that if statement to verify that the result has "shouldEndSession" on it may resolve this bug. Only if that flag is set to true should it resume. That check will certainly fix a different bug, which is that the AudioPlayer should not resume if the session has not been ended.

What I cannot figure out, though, looking through the code, is where the attributes would be getting wiped out here, but likely it happens somewhere in the path of the audioPlayer.resume(). Just not obvious where that occurs.