SAP / ui5-uiveri5

End-to-end testing framework for SAPUI5
Apache License 2.0
120 stars 56 forks source link

uiveri5 job in Jenkins fails #365

Closed alikgarai closed 2 years ago

alikgarai commented 2 years ago

Hello Team,

I am facing one weird issue, where the uiveri5 tests run successfully locally, but when I run on the Jenkins, it fails. And every time it fails for different steps (not consistently failing for one particular step). I have also attached the last 3 test report in the github repo as mentioned below.

Following is the snippet for one such error in Jenkins image

I use control locators with the help of ui5 test recorder tool.

Reference: https://github.tools.sap/I332308/cias-uiveri5

Thank you, Alik

hmanchev commented 2 years ago

Hi Alik,

In the logs you provided I can see that the element is found but is not visible. This can be caused by smaller browser window in your Jenkins execution. The tests can pass locally with bigger browser window as all used elements are visible, but with smaller, they can be hidden (overflow).

In order to be able to further help you, I will need more details about the failure you are receiving. For example screenshot report? Other useful detail would be how you are running the UIveri5 tests in Jenkins - headless browser, browser cloud, Piper step?

Best Regards, Hristo

alikgarai commented 2 years ago

Hi Hristo,

Thanks for the quick reply. I have attached the last 3 screenshot reports in this github link.

In Jenkins, we pull the repo, navigate to the uiveri5 test directory and execute the uiveri5 command, as shown below: image

In the conf file, I didn't specify any browsers, as default is chrome.

Let me know if you need more details or we can even connect over a short call.

Thanks, Alik

hmanchev commented 2 years ago

Hi Alik,

I can't see the images in your report files as you attached only the htmls. Please make sure that the screen size of the browser on your Jenkins executor is at least 1400x1200. If not, please configure to some widely used screen size, for example 1920x1080. Take a look at this doc: https://github.com/SAP/ui5-uiveri5/blob/master/docs/config/config.md#browsers

BR, Hristo

alikgarai commented 2 years ago

Hi Hristo,

I have attached the zip now, with the screenshots. Link

I tried the following approaches to increase the browser size:

image

The issue still persist.

hmanchev commented 2 years ago

Hi Alik,

It seems your configuration is not working as expected. The screenshots you sent are with lower resolution: 1024 × 652. The conf you provided - you can't use maximized: true with combination of browserSize. With this conf the browser will just be maximized to the screen, so if your screen is smaller, you will never get the required browser size.

If you are running on headless chrome, please specify it: $ uiveri5 --browsers=chromeHeadless then remove maximized: true and keep only the browserSize.

Could you please share more details about your Jenkins setup? Are you executing on headless Chrome? Is it local on the Jenkins executor or remote?

Best Regards, Hristo

alikgarai commented 2 years ago

Thank you, Hristo, it worked.

The jenkins job runs on a remote server and I am using headless mode. This is the latest conf which is running fine with full resolution.

image

However, sometimes the Jenkins fails and the screenshot reveals that the page was not completely loaded. Does it mean that I should add explicit wait time for the controls to load properly. I believe uiveri5 takes care of app rendering automatically.

hmanchev commented 2 years ago

Hi Alik,

Is this still relevant?

alikgarai commented 2 years ago

Hi Hristo,

It is resolved, will close the issue.

Thank you, Alik

On Mon, Mar 28, 2022, 8:06 PM Hristo Manchev @.***> wrote:

Hi Alik,

Is this still relevant?

— Reply to this email directly, view it on GitHub https://github.com/SAP/ui5-uiveri5/issues/365#issuecomment-1080735988, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFZKTJSGHPBBTLVMJB5LRATVCG7XDANCNFSM5NANGPCA . You are receiving this because you authored the thread.Message ID: @.***>