Intermittent behaviour of screenshot missing for failed tests in Report Portal

[ztshashi]

The problem

In Report portal screenshots are missing for some failed tests, others have screenshots attached. Below is a sample screenshot of the error.

Environment

• WebdriverIO version: 6.1.14
• wdio-reportportal-reporter version: 7.4.0
• Node.js version: 16.5.0
• WDIO Testrunner and version(mocha\cucumber\jasmine): 6.1.14

Details

The execution log for which screenshot is not attached is below:

Error: function timed out, ensure the promise resolves within 1000 milliseconds
    at Timeout._onTimeout (/app/node_modules/cucumber/lib/user_code_runner.js:76:18)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)

Sample Webdriver Hooks:

afterStep: function (test, context, { error }) {
    if (error) {
      browser.saveScreenshot(failureScreenshotPath);
      screenshot = true;
    }
  },
  afterScenario: function (uri, feature, scenario, result) {
    if (result.status === 'failed' && !screenshot) {
      browser.saveScreenshot(failureScreenshotPath);
    }
  }

WebdriverIO Capabilities:

{
    maxInstances: 2,
    browserName: 'chrome',
    'goog:chromeOptions': {
      args: ['--headless',
        '--window-size=1600,1200']
    }
}

Sample wdio.conf.js to highlight the timeout referred in bug:

exports.config = {
    framework: 'cucumber',
    cucumberOpts: {
        timeout: 20000
    },
}

Code To Reproduce Issue [ Good To Have ]

Configure cucumberOpts timeout to a small number e.g. 1000 in wido.config.js and execute sample cucumber test using webdriverio. The timeout is kept low to cause the function timeout error for which the screenshot is not attached.

[BorisOsipov]

@ztshashi could you please first check that afterStep\afterScenario executed when timeout error appears? e.g. add console.log("foo") to hooks and check console output.

[ztshashi]

@BorisOsipov Sorry for the delayed response, below is what I see when in afterStep:

| [0-0] ====In after step code=== //these are just text I added to locate in execution logs
| [0-0] failureScreenshotPath =<Location of screenshot on the container>/<screenshot-filename>.png
| error = Error: element ("<xpath of web element>") still not clickable after 20000ms
| [0-0] ====end of after step code=== //these are just text I added to locate in execution logs

I am running cucumber tests using webdriverio on docker setup, below is what the execution log for the test.

[chrome 87.0.4280.141 linux #0-0] Verify Application icon (article pages bottom) redirecting properly Hook
[chrome 87.0.4280.141 linux #0-0] function timed out, ensure the promise resolves within 1000 milliseconds
[chrome 87.0.4280.141 linux #0-0] Error: function timed out, ensure the promise resolves within 1000 milliseconds
[chrome 87.0.4280.141 linux #0-0]     at Timeout._onTimeout (/app/node_modules/cucumber/lib/user_code_runner.js:76:18)
[chrome 87.0.4280.141 linux #0-0]     at listOnTimeout (internal/timers.js:549:17)
[chrome 87.0.4280.141 linux #0-0]     at processTimers (internal/timers.js:492:7)

I see the above when I run the tests with cucumberOps timeout as 5000, 10000 in wdio config. Hope this information is helpful.

[BorisOsipov]

@ztshashi 1) can you answer shortly: does afterStep execute when timeout error appears? It is hard to interpret logs output when you can't see the code :) l don't want any confusion or misunderstanding. 2) do you use wdio sync or async? seems sync.

[BorisOsipov]

@ztshashi i can't reproduce issue. see example https://github.com/BorisOsipov/wdio-issues-178 (replace token and rp url in wdio config)

[BorisOsipov]

feel free to reopen or add comment or better repo how to reproduce the issue

[BorisOsipov]

Also I assume if you use node 16 you use ASYNC mode. Please pay attention that async\await is missing in your code snippents.

[ztshashi]

@ztshashi

1. can you answer shortly: does afterStep execute when timeout error appears? It is hard to interpret logs output when you can't see the code :) l don't want any confusion or misunderstanding. Response: With the logging added I see that afterStep is getting executed.
2. do you use wdio sync or async? seems sync. Response: We are using the sync mode, I had also tried an execution by using async and await keyword for afterStep method but I still see instances of screenshots missing when function timeout happens.

Little bit info on the execution environment, I am running tests on a container and the WebdriverIO framework and WebdriverIO reporter is executing in the container, is there anyway i can log what service of reporter is called and the response for the same?

[ztshashi]

Hi Boris,

Good Morning

I have added below response to your queries on the Issue #178

1. can you answer shortly: does afterStep execute when timeout error appears? It is hard to interpret logs output when you can't see the code :) l don't want any confusion or misunderstanding.

Response: With the logging added I see that afterStep is getting executed.

2. do you use wdio sync or async? seems sync.

Response: We are using the sync mode, I had also tried an execution by using async and await keyword for afterStep method but I still see instances of screenshots missing when function timeout happens.

Little bit info on the execution environment, I am running tests on a container and the WebdriverIO framework and WebdriverIO reporter is executing in the container, is there anyway i can log what service of reporter is called and the response for the same?

I was trying to reopen the ticket but couldn't do it. Need your help 

Thanks,

Shashi

---- On Wed, 20 Apr 2022 00:26:38 +0530 Boris Osipov @.***> wrote ----

Also I assume if you use node 16 you use ASYNC mode. Please pay attention that async\await is missing in your code snippents.

— Reply to this email directly, https://github.com/BorisOsipov/wdio-reportportal-reporter/issues/178#issuecomment-1102981398, or https://github.com/notifications/unsubscribe-auth/AWWO7XRE6QDNIMB4VH5Z6T3VF36WNANCNFSM5TWH2ERA. You are receiving this because you were mentioned.

[BorisOsipov]

@ztshashi see https://github.com/BorisOsipov/wdio-reportportal-reporter/issues/178#issuecomment-1102979370

can you reproduce the issue by this repo?

[ztshashi]

@BorisOsipov sorry for the delayed response, I tried changing my afterStep for keywords "async" and "await" below is the same code I used to capture screenshot. I still see instances of missing screenshot. browser.saveScreenshot(<variable having filepath>);

I will try this repo and share the results thanks