search

GoogleChromeLabs / chromium-bidi

Implementation of WebDriver BiDi for Chromium
https://googlechromelabs.github.io/chromium-bidi/
Apache License 2.0
85 stars 28 forks source link

Implement: `browsingContext.captureScreenshot` #514

Open thiagowfx opened 1 year ago

thiagowfx commented 1 year ago

Tracking

thiagowfx commented 1 year ago

See also: https://github.com/web-platform-tests/wpt/pull/37454/

sadym-chromium commented 1 year ago

WPT headful test is failing. I temporary disabled it in https://github.com/GoogleChromeLabs/chromium-bidi/pull/548

sadym-chromium commented 1 year ago

DevTools has functionality to capture node screenshot. What it does is: Call callFunctionOn(node) with script:

() => {
   const e = this.getBoundingClientRect(),
   t = this.ownerDocument.documentElement.getBoundingClientRect();
   return JSON.stringify({
      x: e.left - t.left,
      y: e.top - t.top,
      width: e.width,
      height: e.height,
      scale: 1.0
  });
}

And call Page.captureScreenshot with the following params:

{
  "format": "png",
  "quality": 100,
  "fromSurface": true,
  "captureBeyondViewport": true,
  "clip": { 
      RECEIVED_VALUES
  }
}
sadym-chromium commented 1 year ago

The comment above could help implementing the nested frame screenshot.

thiagowfx commented 1 year ago

References:

thiagowfx commented 1 year ago

@sadym-chromium:

const e = this.getBoundingClientRect() seems to be

const metrics = await this.#cdpTarget.cdpClient.sendCommand(
  'Page.getLayoutMetrics'
).cssContentSize;

Do you know how to retrieve "t"?

thiagowfx commented 1 year ago

Debugging:

async captureScreenshot(): Promise<BrowsingContext.CaptureScreenshotResult> {
    // XXX: Either make this a proposal in the BiDi spec, or focus the
    // original tab right after the screenshot is taken.
    // The screenshot command gets blocked until we focus the active tab.
    await this.#cdpTarget.cdpClient.sendCommand('Page.bringToFront'); // window.focus() also works

    const docRect = await this.#cdpTarget.cdpClient.sendCommand(
      'Runtime.callFunctionOn',
      {
        functionDeclaration: `() => {
        const docRect = window.documentElement.getBoundingClientRect();
        return JSON.stringify({
          x: docRect.left,
          y: docRect.top,
        });
      }`,
        executionContextId: this.#defaultRealm.executionContextId,
      }
    );
    const {result: docRectResult} = docRect;
    console.log(docRectResult);

    const metrics = await this.#cdpTarget.cdpClient.sendCommand(
      'Page.getLayoutMetrics'
    );
    // or maybe cssLayoutViewport
    const {cssContentSize: viewport} = metrics;

    debugger;

    const [result] = await Promise.all([
      this.#cdpTarget.cdpClient.sendCommand('Page.captureScreenshot', {
        format: 'png', // XXX: add more formats: jpeg,webp, then add quality for jpeg
        captureBeyondViewport: true,
        clip: {...viewport, scale: 1.0},
      }),
    ]);
    return {
      result: {
        data: result.data,
      },
    };
  }
thiagowfx commented 1 year ago

More debugging:

async captureScreenshot(): Promise<BrowsingContext.CaptureScreenshotResult> {
    // XXX: Either make this a proposal in the BiDi spec, or focus the
    // original tab right after the screenshot is taken.
    // The screenshot command gets blocked until we focus the active tab.
    await this.#cdpTarget.cdpClient.sendCommand('Page.bringToFront'); // window.focus() also works

    const docRect = await this.#cdpTarget.cdpClient.sendCommand(
      'Runtime.callFunctionOn',
      {
        functionDeclaration: `() => { return JSON.stringify(globalThis.document.documentElement.getBoundingClientRect()); }`,
        executionContextId: this.#defaultRealm.executionContextId,
      }
    );
    console.log(docRect);

    const metrics = await this.#cdpTarget.cdpClient.sendCommand(
      'Page.getLayoutMetrics'
    );
    // or maybe cssLayoutViewport
    const {cssContentSize: viewport} = metrics;

    debugger;

    const [result] = await Promise.all([
      this.#cdpTarget.cdpClient.sendCommand('Page.captureScreenshot', {
        format: 'png', // XXX: add more formats: jpeg,webp, then add quality for jpeg
        captureBeyondViewport: true,
        clip: {...viewport, scale: 1.0},
      }),
    ]);
    return {
      result: {
        data: result.data,
      },
    };
  }
thiagowfx commented 7 months ago

Implement scrollIntoView

This one can be closed, as it was removed from the BiDi spec.