aws / amazon-chime-sdk-js

A JavaScript client library for integrating multi-party communications powered by the Amazon Chime service.
Apache License 2.0
701 stars 473 forks source link

Invoke MediaStreamBrokerObservers in the next event loop tick #2725

Closed simmkyu closed 1 year ago

simmkyu commented 1 year ago

Issue #: https://github.com/aws/amazon-chime-sdk-js/issues/2709

Conditions to occur:

Whenever there's a change in the audio input, the SDK calls the RTCRtpSender: replaceTrack API in the browser to swap out the track that's currently in use.

  1. Join a meeting with Web Audio disabled.
  2. Call startAudioInput with a new audio input.
    • Reusing the existing microphone is okay.
  3. In the same JavaScript code block, call startVideoInput with the MediaStream object.
    • Calling it with a non-MediaStream object is okay.
  4. In the same JavaScript code block, call startLocalVideoInput.
    • Calling it after a 0 ms delay is okay. JavaScript executes the code in the next cycle.

Root cause:

If the above conditions are satisfied, the SDK attempts to call the RTCRtpSender: replaceTrack API twice.

  1. At the end of the startAudioInput API.
  2. During the update action triggered by the startLocalVideoTile API.

The SDK should call the RTCRtpSender: replaceTrack in sequence, first in (1) followed by (2). However, in the stated conditions, step (2) happens after step (1).

Description of changes:

Testing:

  1. Assume that you have at least one microphone and camera configured in your system.
  2. Open the Chime SDK Serverless Demo in your browser.
  3. Create and join a Chime SDK meeting.
  4. Open the browser console and execute the following code:

    (async () => {
      const stream = await navigator.mediaDevices.getUserMedia({
        audio: true,
        video: true,
      });
    
      const audioTracks = stream.getAudioTracks();
      const audioTrack = audioTracks[0];
    
      const videoTracks = stream.getVideoTracks();
      const videoTrack = videoTracks[0];
    
      // Start audio!
      const audioStream = new MediaStream([audioTrack]);
      await app.audioVideo.startAudioInput(audioStream);
    
      // Start video!
      const videoStream = new MediaStream([videoTrack]);
      await app.audioVideo.startVideoInput(videoStream);
      app.audioVideo.startLocalVideoTile();
    })();
  5. Ensure that the Chime SDK does not display the SendingAudioFailed event in the browser console. Other attendees should not have any issues hearing and seeing you.

Can these tested using a demo application? Please provide reproducible step-by-step instructions.

Yes. See the Testing section.

Checklist:

  1. Have you successfully run npm run build:release locally? Yes.

  2. Do you add, modify, or delete public API definitions? If yes, has that been reviewed and approved? No.

  3. Do you change the wire protocol, e.g. the request method? If yes, has that been reviewed and approved? No.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.