What does capturehandlechange event do when cross documentation navigation happen?

[xuesichao]

Hi, I see in the blog Better tab sharing with Capture Handle :

Monitor CaptureHandle changes by listening to "capturehandlechange" events on a MediaStreamTrack object. Changes happen when:

• The captured web app calls navigator.mediaDevices.setCaptureHandleConfig().
• A cross-document navigation occurs in the captured web app.

I found it not clear on what will be the value of event.target.getCaptureHandle() when cross-document navigation happens.

// When cross-document navigation happens
videoTrack.addEventListener('capturehandlechange', event => {
  captureHandle = event.target.getCaptureHandle();
  // What's the value of captureHandle?
});

Could you help clarify this? It seems the captureHandle will be null in this case? And when does this event gets triggered exactly? Is there a specific DOM event? Thank you.

[eladalon1983]

I think the spec can clarify this topic - see here. Let me know if that helped.

[xuesichao]

@eladalon1983 I checked the spec, it is helpful but not fully answer my question. Our use case is that we want to send the URL of the captured website via handle so we can do some URL restriction, if URL is not allowed we want to stop the content share. We also check the existence of the signed captureHandle to tell if it's an allowed origin or an external origin.

My problem is that when refreshing the same page, it's a cross-document navigation and the captureHandle is set to null by browser (my guess) ,even if I manually setCaptureHandleConfig in the of the .html file. Could you help if you know any way to avoid this null value on page refresh?

Implementation:

// Captured side, this is in a .js file that is imported via <script> in <header> of the .html file
(function () {
  if (!('setCaptureHandleConfig' in navigator.mediaDevices)) {
    console.log('setCaptureHandleConfig not supported');
    return;
  }

  if (!('navigation' in window)) {
    console.log('navigation not supported');
    return;
  }

  const setCaptureHandleConfig = (url: URL) => {
    const { origin, pathname } = url;
    // @ts-ignore
    navigator.mediaDevices.setCaptureHandleConfig({
      handle: JSON.stringify({ identifier: 'signature-xxxx', url: origin + pathname }),
      exposeOrigin: true,
      permittedOrigins: [origin],
    });
  };

  const url = new URL(window.location.href);
  setCaptureHandleConfig(url);

  const handleNavigate = (event: { destination: { url: string } }) => {
    const url = new URL(event.destination.url);
    setCaptureHandleConfig(url);
  };

  // @ts-ignore
  window.navigation.addEventListener('navigate', handleNavigate);

  window.addEventListener('unload', () => {
    // @ts-ignore
    window.navigation.removeEventListener('navigate', handleNavigate);
  });
})();

// Capturing side
stream = await navigator.mediaDevices.getDisplayMedia(constraints);
const [videoTrack] = stream.getVideoTracks();
const captureHandle = videoTrack.getCaptureHandle();
// Initial URL upon start screen capture
const { url } = JSON.parse(captureHandle.handle);

if (isUrlAllowed(url)) {
  videoTrack.addEventListener('capturehandlechange', (event) => {
    const captureHandle = event.target.getCaptureHandle();
    // Latest URL on navigation change
    const { url } = JSON.parse(captureHandle.handle);
    if (!isUrlAllowed(url)) {
      // Stop the screen capture
      // My problem is that when refreshing a allowed URL/origin, the captureHandle will first be `null`
    }
  });
  // Start the screen capture
} else {
  // Do not start the screen capture
}

[eladalon1983]

The capture handle is indeed reset, and you'll get no capture handle until the refreshed page loads to the point where it sets it again. I don't think we can avoid this atm, because we don't really know if the captured page will ever call setCaptureHandleConfig() again. But I believe you have a work-around available:

• Upon observing the change of capture handle from a permitted origin to null, pause remote transmission of the video.
• Hold off on any user-facing messaging for a few seconds.
• If the refreshed page ends up setting the permitted origin - great, it's a refresh and you can resume remote transmission of the stream.
• Otherwise, display some message to the user.

It's an imperfect solution - it will delay messages when they're needed, and occasionally it will show messages right before the page loads far enough to set the capure handle - but that's what you can do with existing APIs. If you try this out, I would love to hear the empirical evidence of how well it worked.

Btw, you might be concerned that not all frames will be supressed, because the event might fire and be handled after some disallowed frames are captured and transmitted remotely. I had filed this proposal for it in the past. Since then, @tovepet has taken over this endeavor. I bet she'd love to get in touch with you and see if the solution she's working on could address your use case as well.

[xuesichao]

The work-around we have right now is very similar to your suggestion, the only difference is that instead of pausing remote transmission, we set the track.enabled to disabled when captureHandle is null, so the video stream is blank/black.

stream.getTracks().forEach(track=> track.enabled = false)

The capture handle is indeed reset, and you'll get no capture handle until the refreshed page loads to the point where it sets it again.

That's exactly what I wanted to confirm. Not sure if this is a common knowledge for web developer when cross-document navigation happens, it seems so. It might be helpful to add this behavior to the blog here: https://developer.chrome.com/docs/web-platform/capture-handle#:~:text=A%20cross%2Ddocument%20navigation%20occurs%20in%20the%20captured%20web%20app.

For captured content switching. Since we manually set the tract.enabled to false, the result is quite similar to auto-pause feature you are proposing. It will be way more reliable if we can detect it from dedicated browser APIs.

@eladalon1983 Thank you so much for your insights, you are very friendly and thoughtful.