Expected DeviceChangeObserver Callbacks not triggered when revoking device permissions from browser

[mattcarnovale-streem]

What happened and what did you expect to happen?

When a user revokes permissions to their camera or microphone using the browser settings, we expect the videoInputStreamEnded and audioInputStreamEnded callbacks offered through the DeviceChangeObserver to be invoked. We were able to assert this behavior using the Chrome browser on a Macbook Pro and Android mobile device (Galaxy S20 FE). However, it was observed that the same callbacks were not being invoked under the same conditions as well as with Safari when using an iPhone (iPhone 13 mini & iPhone 12).

As a reference, here are the browser settings being used to revoke device permissions:

Have you reviewed our existing documentation?

• [X] GitHub issues
• [X] FAQs page
• [X] README page
• [X] Feature-specific guides

Reproduction steps

Using an iPhone device:

• Allow permissions to camera and microphone when joining a Chime meeting
• Revoke permissions to either the camera or microphone while in the meeting
• Observe that the videoInputStreamEnded and audioInputStreamEnded callbacks are not invoked

Amazon Chime SDK for JavaScript version

3.521.0

What browsers are you seeing the problem on?

Chrome, Safari

Browser version

Chrome: 126.0.6478.54, Safari: iOS 17.5.1

Meeting and Attendee ID Information.

No response

Browser console logs

Both logs are recording start of meeting, revoking camera and then revoking microphone.

Chrome:

Safari:

[michhyun1]

Thats interesting - it could be a webkit specific implementation.

This is where we call that observer function: https://github.com/aws/amazon-chime-sdk-js/blob/90baf9995f10fc86c1c39b91e0c2ea926138fd82/src/devicecontroller/DefaultDeviceController.ts#L1215-L1224

On the ended event for that media stream track. Its possible that Safari doesnt trigger that event when the permissions are turned off, however, the docs mention here that the ended event should be called when permissions are revoked. https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/ended_event#:~:text=The%20user%20revoked%20the%20permissions%20needed%20for%20the%20data%20to%20be%20sent.

[michhyun1]

Also - what does the screen shot show? MAybe I'm misisng it but is the videoInputStreamEnded callback function printing something in the Chrome screenshot?

[mattcarnovale-streem]

Also - what does the screen shot show? MAybe I'm misisng it but is the videoInputStreamEnded callback function printing something in the Chrome screenshot?

Oh, sorry if the screenshot was misleading (and way larger than I thought it would be). There was a lot of confusion when communicating what exactly was meant by "browser device settings" and how to revoke the permissions to trigger the expected callback, so I included a screenshot of the controls I was using when testing with Chrome.

[michhyun1]

I was talking about the chrome/safari logs screenshot haha - what is the log that is expected to be printed on videoInputStreamEnded?

[mattcarnovale-streem]

Ohh, those screenshots 😆 Got it!

So here are the logs when using my laptop and the Chrome browser:

Upon denying the camera permissions:

Upon denying the microphone permissions:

The last INFO and WARN log coming from sdk-chime-video are coming from our videoInputStreamEnded callback implementation. We aren't see these logs or that WARN from chime-sdk-js when revoking the device permission on an iPhone.

[michhyun1]

I've submitted a bug for webkit, I'm not sure why that event is not fired https://bugs.webkit.org/show_bug.cgi?id=275918

[xuesichao]

It seems once you grant permission to Safari, it only has pause/resume option. Not like Chrome where you can deny it. For Safari, the ended callback is not invoked in such case.

You could verify this behavior by using below code pen: https://codepen.io/xuesichao/pen/abrrLzN

I also share the code here:

if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
  const constraints = {
    audio: true,
    video: true
  };

  navigator.mediaDevices.getUserMedia(constraints)
    .then((stream) => {
      console.log('Media stream obtained successfully:', stream);

      // Create a container div for the overlay
      const overlayContainer = document.createElement('div');
      overlayContainer.style.position = 'absolute';
      overlayContainer.style.top = '10px';
      overlayContainer.style.left = '10px';
      overlayContainer.style.backgroundColor = 'rgba(0, 0, 0, 0.7)';
      overlayContainer.style.padding = '10px';
      overlayContainer.style.borderRadius = '5px';
      overlayContainer.style.display = 'flex';
      overlayContainer.style.flexDirection = 'column';
      overlayContainer.style.alignItems = 'flex-start';
      document.body.appendChild(overlayContainer);

      // Create and attach the video element to the container
      const videoElement = document.createElement('video');
      videoElement.srcObject = stream;
      videoElement.autoplay = true;
      videoElement.style.width = '300px';
      videoElement.style.borderRadius = '5px';
      overlayContainer.appendChild(videoElement);

      // Create and attach the message element to the container
      const messageElement = document.createElement('p');
      messageElement.style.color = 'white';
      messageElement.style.marginTop = '10px';
      overlayContainer.appendChild(messageElement);

      // Add event listeners for ended events
      stream.getTracks().forEach((track) => {
        track.addEventListener('ended', () => {
          const message = `Track ended: ${track.kind}`;
          console.log(message);
          messageElement.textContent += message + '\n';
        });
      });
    })
    .catch((error) => {
      console.error('Error accessing media devices:', error);

      // Display error message in the document
      const overlayContainer = document.createElement('div');
      overlayContainer.style.position = 'absolute';
      overlayContainer.style.top = '10px';
      overlayContainer.style.left = '10px';
      overlayContainer.style.backgroundColor = 'rgba(255, 0, 0, 0.7)';
      overlayContainer.style.padding = '10px';
      overlayContainer.style.borderRadius = '5px';
      document.body.appendChild(overlayContainer);

      const errorMessage = document.createElement('p');
      errorMessage.style.color = 'white';
      errorMessage.textContent = `Error accessing media devices: ${error.message}`;
      overlayContainer.appendChild(errorMessage);
    });
} else {
  console.error('getUserMedia is not supported in this browser.');

  // Display a message if getUserMedia is not supported
  const overlayContainer = document.createElement('div');
  overlayContainer.style.position = 'absolute';
  overlayContainer.style.top = '10px';
  overlayContainer.style.left = '10px';
  overlayContainer.style.backgroundColor = 'rgba(255, 0, 0, 0.7)';
  overlayContainer.style.padding = '10px';
  overlayContainer.style.borderRadius = '5px';
  document.body.appendChild(overlayContainer);

  const notSupportedMessage = document.createElement('p');
  notSupportedMessage.style.color = 'white';
  notSupportedMessage.textContent = 'getUserMedia is not supported in this browser.';
  overlayContainer.appendChild(notSupportedMessage);
}

[mattcarnovale-streem]

I see, so the documentation should probably be updated to indicate this is not supported in Safari as a result of the behavior you've verified. Shouldn't it?

Was this issue closed because a bug was submitted for webkit? The comment above doesn't address the fact that the callback is not triggerd when using Chrome with an iPhone which is part of the issue reported.

[xuesichao]

Hi @mattcarnovale-streem, When you revoke the device permission using iOS Safari or iOS Chrome, the MediaStreamTrack does not end. This behavior is specific to these browsers, and as a result, the DeviceChangeObserver is not triggered in such cases. The SDK is working properly so I close this issue.

The comment above doesn't address the fact that the callback is not triggerd when using Chrome with an iPhone which is part of the issue reported.

iOS Chrome also does not support this feature, as it uses the same WebKit engine as iOS Safari. You can verify this behavior using the provided CodePen example.

@mattcarnovale-streem Could you elaborate your use case a bit more regarding the need of revoking the device permission? Browser has limited support regarding device permission check.

You may want to use browser API await navigator.permissions.query({ name: "camera" }) to check if browser has permission to the camera. This API is not supported by Firefox, and only available in Chrome and Safari 16.

• https://developer.mozilla.org/en-US/docs/Web/API/Permissions/query
• https://developer.mozilla.org/en-US/docs/Web/API/Permissions_API

[eric-carlson]

What API does Chime call when a user revokes permissions? IOW, what is supposed to stop the track?