Support for an screencast method

[Darkensses]

Hello. I wonder if it's planning to add a similar method to screencast the frames from a session/test, similar to the CDP method Page.startScreencast.

After get/save the frames, it's possible to process and convert them in a video file. Useful for reporting when run the test in a headless browser and see what happened.

I know that the purpose of the webdriver-bidi protocol is not to make a CDP clone, but maybe adding this feature could be very interesting. I also think that this CDP feature is not very complex to implement in a real project, so maybe it's worth doing the effort to add it to the set of tool you already have.

CDP example

cdp.on('message', msg => {
  const obj = JSON.parse(msg);
  if(obj.method === 'Page.screencastFrame') {
    ws.send(JSON.stringify({
      id: id++,
      method: 'Page.screencastFrameAck',
      params: { sessionId: obj.params.sessionId },
    }));
    // Do your stuff...
  }
});

cdp.send(JSON.stringify({
  id: id++,
  method: 'Page.startScreencast',
  params: {
    maxWidth: 800,
    maxHeight: 600,
  }
}));

https://github.com/w3c/webdriver-bidi/assets/23271049/401008e8-9e17-4eb0-beb3-00168e39c872

video created with FFMPEG

Please, let me know if this can be planned in a future and feel free to correct me if this is not the right place to ask or if you need more context or info.

Thanks! :)

[whimboo]

Thank you @Darkensses for your feature request. It sounds useful to have and as such I've added it to the list of future APIs to discuss.

If you have other APIs, which would help you to get automation done with WebDriver BiDi, please do not hesitate to file other issues for feature requests.

[OrKoN]

I think screencasting should be based on https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia instead of the current CDP's approach.

[nir-tal-talkspace]

I think this is a critical feature to create videos in automation @whimboo I use the CDP version devTools.addListener(Page.screencastFrame(), screencastFrame -> Try.withResources(() -> new FileOutputStream(Constants.VIDEO_DIRECTORY + File.separator + System.currentTimeMillis() + ".jpeg")) .of(stream -> { stream.write(Base64.getDecoder().decode(screencastFrame.getData())); // send ack otherwise frames stop to arrive devTools.send(Page.screencastFrameAck(screencastFrame.getSessionId())); return true; }));