Doc Detective hangs when using cropping feature of screenshot when element is not fully onscreen

[ggray-cb]

So, I began testing Doc Detective with our product, and have hit a hang that seem to happen with screenshots using the cropping feature. I'm running on MacOS, talking to an instance of our product's web interface running locally via docker. Here's the test .json file I have:

{
  "contexts": [
    {
      "app": {
        "name": "firefox",
        "options": {
          "headless": false
        }
      },
      "platforms": [
        "windows",
        "mac",
        "linux"
      ]
    }
  ],
  "tests": [
    {
      "id": "verify_web_console_up",
      "description": "Test that the login page actually exists",
      "steps": [
        {
          "description": "See if server Web UI is up.",
          "action": "checkLink",
          "url": "http://localhost:8091/",
          "statusCodes": [
            200,
            201,
            202
          ]
        }
      ]
    },
    {
      "id": "create_bucket_demo",
      "description": "Walk through setting up a bucket via the UI.",
      "steps": [
        {
          "description": "Navigate to login page.",
          "action": "goTo",
          "url": "http://localhost:8091/"
        },
        {
          "description": "Enter username",
          "action": "find",
          "selector": "#auth-username-input",
          "moveTo": true,
          "click": true,
          "typeKeys": "Administrator"
        },
        {
          "description": "Enter password",
          "action": "find",
          "selector": "#auth-password-input",
          "moveTo": true,
          "click": true,
          "typeKeys": "password"
        },
        {
          "description": "Click Sign In",
          "action": "find",
          "selector": ".panel-footer > button",
          "moveTo": true,
          "click": true
        },
        {
          "description": "Take screenshot",
          "action": "saveScreenshot",
          "path": "results.png",
          "overwrite": true
        },
        {
          "description": "Click Buckets",
          "action": "find",
          "selector": ".nav-sidebar > a:nth-child(3)",
          "matchText": "Buckets",
          "moveTo": true,
          "click": true
        },
        {
          "action": "wait"
        },
        {
          "description": "Click Add Bucket",
          "action": "find",
          "selector": ".header-controls > a:nth-child(1)",
          "click": true
        },
        {
          "action": "wait"
        },
        {
          "description": "Take screenshot of add bucket dialog",
          "action": "saveScreenshot",
          "crop": {
            "selector": ".dialog-med",
            "padding": 0
          },
          "path": "add-bucket-dialog-1.png",
          "overwrite": true
        },
        {
          "action": "wait"
        },
        {
          "description": "Enter name for new bucket",
          "action": "find",
          "selector": "input.ng-pristine",
          "moveTo": true,
          "click": true,
          "typeKeys": "demo-bucket"
        },
        {
          "description": "Expand advanced settings",
          "action": "find",
          "selector": ".disclosure",
          "moveTo": true,
          "click": true
        },
        {
          "description": "Scroll to Conflict resolution to prep for screenshot",
          "action": "find",
          "selector": "label",
          "matchText": "Conflict Resolution",
          "moveTo": true
        },
        {
          "description": "Take screenshot of advanced bucket settings pt1",
          "action": "saveScreenshot",
          "crop": {
            "selector": ".forms > div:nth-child(6)",
            "padding": 0
          },
          "path": "add-bucket-dialog-advanced-settings-p1.png",
          "overwrite": true
        },
        {
          "action": "wait"
        }
      ]
    }
  ]
}

Doc Detective hangs when taking one of the last two screenshots that do cropping. The selector is a modal dialog box that appears after the Add Bucket link is clicked. In both cases, the screenshot is taken, and appears on disk, but Doc Detective goes no further. If I have the browser window in the foreground, it closes eventually, but I need to Ctrl+C Doc Detective on the command line to get it to exit.

Here's the end of the run:

2024-11-21T20:32:14.916Z INFO webdriver: DATA {
  script: 'return (() => window.devicePixelRatio).apply(null, arguments)',
  args: []
}
2024-11-21T20:32:14.924Z INFO webdriver: RESULT 1
2024-11-21T20:32:14.924Z INFO webdriver: COMMAND findElement("css selector", ".forms > div:nth-child(6)")
2024-11-21T20:32:14.924Z INFO webdriver: [POST] http://0.0.0.0:4723/session/1e702e7c-4f6c-428f-9ae4-ae76a0b4f0ec/element
2024-11-21T20:32:14.924Z INFO webdriver: DATA { using: 'css selector', value: '.forms > div:nth-child(6)' }
2024-11-21T20:32:14.930Z INFO webdriver: RESULT {
  ELEMENT: 'f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212',
  'element-6066-11e4-a52e-4f735466cecf': 'f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212'
}
2024-11-21T20:32:14.931Z INFO webdriver: COMMAND getElementRect("f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212")
2024-11-21T20:32:14.931Z INFO webdriver: [GET] http://0.0.0.0:4723/session/1e702e7c-4f6c-428f-9ae4-ae76a0b4f0ec/element/f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212/rect
2024-11-21T20:32:14.937Z INFO webdriver: RESULT { height: 671, width: 498, x: 343.5, y: 542.375 }
2024-11-21T20:32:14.937Z INFO webdriver: COMMAND getElementRect("f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212")
2024-11-21T20:32:14.937Z INFO webdriver: [GET] http://0.0.0.0:4723/session/1e702e7c-4f6c-428f-9ae4-ae76a0b4f0ec/element/f.B219E63B9B0F293F3FF48835A6940353.d.33A4FD653F2D7698836E0D043D38F868.e.212/rect
2024-11-21T20:32:14.943Z INFO webdriver: RESULT { height: 671, width: 498, x: 343.5, y: 542.375 }

I've tried both Firefox and Chrome. Firefox hangs at the second screenshot ("Take screenshot of add bucket dialog"). Chrome hangs at the third. What I suspect is the issue is that the element for these cropped screenshots hit the bottom edge of the browser window. When run in the foreground, I can see the dialog is large enough to go offscreen on Firefox because its scaling seems to be larger. In Chrome, the second screenshot succeeds because the dialog box is fully onsreen. Chrome hangs on the third screenshot because clicking the dropdown expands the dialog offscreen.

[dosubot[bot]]

Hello @ggray-cb! I'm here to assist you with any bugs, questions, or contributions. Let's work together to resolve the issue you're facing.

Doc Detective currently does not handle elements that are partially offscreen when taking cropped screenshots, which can lead to issues like hanging during test execution. The framework calculates the bounding rectangle of the element for cropping, but there is a known limitation where error handling for out-of-bounds conditions is not yet implemented ^[1].

To work around this issue, you can try ensuring that the elements you want to capture are fully visible on the screen before taking the screenshot. This might involve adjusting the browser window size or scrolling the element into view programmatically before executing the screenshot step.

To continue talking to Dosu, mention @dosu.

[hawkeyexl]

Hi @ggray-cb. Good catch on this edge case. Right now we don't have any logic to forcibly scroll the element into the viewport before taking the screenshot, but it seems like that's something we'll have to add to remedy this.

In the meantime, you might do a find action for the element, because that will normally scroll it into view. Even then, you might run into issues with particularly large elements, but this should mitigate the short-term issue while we work on a long-term fix.

[hawkeyexl]

I'll move this to the core repo once I'm back on desktop.

[dosubot[bot]]

The issue you're experiencing with Doc Detective hanging during the cropping feature of screenshots when elements are not fully onscreen is likely related to the lack of error handling for out-of-bounds conditions in the cropping functionality. Currently, Doc Detective does not explicitly handle elements that are not fully visible, which can lead to errors if the cropping area extends beyond the bounds of the screenshot ^[1].

To address this, you might consider the following workarounds:

1. Ensure Elements are Fully Visible: Before taking a screenshot, make sure the element is fully visible within the viewport. You can try manually scrolling the element into view or adjusting the browser window size to accommodate the entire element.

2. Modify Test Steps: Adjust your test steps to ensure that the elements are fully onscreen before taking a screenshot. This might involve adding additional steps to scroll or resize the window.

3. Custom Error Handling: If you have access to modify the code, consider adding custom error handling to manage out-of-bounds conditions when cropping screenshots.

These steps should help mitigate the issue until a more permanent solution is implemented in Doc Detective.

To continue talking to Dosu, mention @dosu.