ui5-community / wdi5

official UI5 end-to-end test framework for UI5 web-apps. wdi5 = Webdriver.IO + UI5 Test API
https://ui5-community.github.io/wdi5/
Apache License 2.0
102 stars 43 forks source link

Fiori Elements test is failing #290

Closed T1mey closed 2 years ago

T1mey commented 2 years ago

Hello,

we're testing a FE app and have a simple testcase which is failing.

Given.onTheMainPage.onTable().iExecuteCreate()
Then.onTheDetailPage.iSeeThisPage().and.iSeeObjectPageInEditMode()

If we're executing it a failure is occouring:

[chrome 102.0.5005.115 windows #0-0] 1) FE basics should see an object page
[chrome 102.0.5005.115 windows #0-0] javascript error: Cannot read properties of undefined (reading 'waitForUI5')
JavaScript stack:
TypeError: Cannot read properties of undefined (reading 'waitForUI5')
    at eval (eval at executeAsyncScript (:552:26), <anonymous>:5:14)
    at eval (eval at executeAsyncScript (:552:26), <anonymous>:17:8)
    at eval (eval at executeAsyncScript (:552:26), <anonymous>:17:33)
    at executeAsyncScript (<anonymous>:552:47)
    at <anonymous>:567:29
    at callFunction (<anonymous>:446:22)
    at <anonymous>:460:23
    at <anonymous>:461:3
  (Session info: chrome=102.0.5005.115)
[chrome 102.0.5005.115 windows #0-0] javascript error: javascript error: Cannot read properties of undefined (reading 'waitForUI5')
[chrome 102.0.5005.115 windows #0-0] JavaScript stack:
[chrome 102.0.5005.115 windows #0-0] TypeError: Cannot read properties of undefined (reading 'waitForUI5')     
[chrome 102.0.5005.115 windows #0-0]     at eval (eval at executeAsyncScript (:552:26), <anonymous>:5:14)      
[chrome 102.0.5005.115 windows #0-0]     at eval (eval at executeAsyncScript (:552:26), <anonymous>:17:8)      
[chrome 102.0.5005.115 windows #0-0]     at eval (eval at executeAsyncScript (:552:26), <anonymous>:17:33)     
[chrome 102.0.5005.115 windows #0-0]     at executeAsyncScript (<anonymous>:552:47)
[chrome 102.0.5005.115 windows #0-0]     at <anonymous>:567:29
[chrome 102.0.5005.115 windows #0-0]     at callFunction (<anonymous>:446:22)
[chrome 102.0.5005.115 windows #0-0]     at <anonymous>:460:23
[chrome 102.0.5005.115 windows #0-0]     at <anonymous>:461:3
[chrome 102.0.5005.115 windows #0-0]   (Session info: chrome=102.0.5005.115)
[chrome 102.0.5005.115 windows #0-0]     at getErrorFromResponseBody (C:\develop\javascript\onboarding-tool-cap\node_modules\webdriver\build\utils.js:197:12)
[chrome 102.0.5005.115 windows #0-0]     at NodeJSRequest._request (C:\develop\javascript\onboarding-tool-cap\node_modules\webdriver\build\request\index.js:166:60)
[chrome 102.0.5005.115 windows #0-0]     at processTicksAndRejections (node:internal/process/task_queues:96:5) 
[chrome 102.0.5005.115 windows #0-0]     at async Browser.wrapCommandFn (C:\develop\javascript\onboarding-tool-cap\node_modules\@wdio\utils\build\shim.js:137:29)
[chrome 102.0.5005.115 windows #0-0]     at async Browser.wrapCommandFn (C:\develop\javascript\onboarding-tool-cap\node_modules\@wdio\utils\build\shim.js:137:29)
[chrome 102.0.5005.115 windows #0-0]     at async emptyQueue (C:\develop\javascript\onboarding-tool-cap\node_modules\wdio-ui5-service\client-side-js\testLibrary.js:33:12)

Seems that the error is coming from the second line, because a draft record is created.

Another question is when to use which object (Given, When and Then) .... is there a more detailed documentation for FE?

Siolto commented 2 years ago

Hi,

could you maybe share you're whole spec file, that we can see that you initialized the FE test library the right way. Unfortunately the documentation about the integration of the test library in wdi5 is not merged yet but there is already a PR available.

In wdi5 itself we do not use the Given(arrangements), When(actions), Then(assertions) pattern. This comes from OPA5 itself. Maybe you can have a more detailed look here.

Best regards, Simon

T1mey commented 2 years ago

This is our configuration:

const { join } = require("path")

exports.config = {
    wdi5: {
        screenshotPath: join("app", "onboarding_tool", "webapp", "wdi5-test", "__screenshots__"),
        logLevel: "verbose", // error | verbose | silent
        url: "/tool/webapp/index.html",
        waitForUI5timeout: 30000,
    },
    //// wdio runner config
    specs: [join("app", "tool", "webapp", "wdi5-test", "**/*.test.js")],
    // Patterns to exclude.
    exclude: [],
    //// capabilities ("browser") config
    maxInstances: 10,
    capabilities: [
        {
            // overwrite global "maxInstances"
            maxInstances: 5,
            browserName: "chrome",
            acceptInsecureCerts: true,
            "goog:chromeOptions": {
                args:
                    process.argv.indexOf("--headless") > -1
                        ? ["window-size=1440,800", "--headless"]
                        : process.argv.indexOf("--debug") > -1
                        ? ["window-size=1920,1280", "--auto-open-devtools-for-tabs"]
                        : ["window-size=1440,800"]
            }
        }
    ],
    //// test config
    // Level of logging verbosity: trace | debug | info | warn | error | silent
    logLevel: "error",
    bail: 0,
    baseUrl: "http://localhost:4004",

    waitforTimeout: 10000,
    connectionRetryTimeout: 120000,
    connectionRetryCount: 3,

    services: ["chromedriver", "ui5"],

    framework: "mocha",
    mochaOpts: {
        ui: "bdd",
        timeout: 120000
    },
    reporters: ["spec"]
}
Siolto commented 2 years ago

At first look your configuration looks fine. Did you initialized the test library the same way we did it in one of our examples?

T1mey commented 2 years ago

Yes, the same way...

Could you add that in your example:

Given.onTheMainPage.onTable().iExecuteCreate()
Then.onTheDetailPage.iSeeThisPage().and.iSeeObjectPageInEditMode()

Would do it ... but I'm not able to run your test ...

npm run test:fe-app

Error: Error: Cannot find module 'C:\Users\user\Documents\wdi5\node_modules\wdio-ui5-service\dist\index.js'. Please verify that the package.json has a valid "main" entry

Siolto commented 2 years ago

Hi,

have you tried running npm run build first?

just replaced our first test with your coding image And it seems to work: image

T1mey commented 2 years ago

Thanks a lot !!!

Yes .. the npm run build didi it ... I'm able to run your samples ... So I will try to compare the configuration & execution ... as our apps are similar this sould solve the problems...

Let's see... I will update onthe results.

T1mey commented 2 years ago

@Siolto So I had a deeper look into the configuration ... So the difference is that we are starting the entire CAP framework by running

cds run --profile production

and afterwards

wdio run wdio.conf.js

I would assume that this is not a problem?

But maybe another fact.... we are not using the cflp sandbox. Instead we're running the direct url configured in wdio.conf.js

baseUrl: "http://user:somepwd@localhost:4004",

So this opens the application directly on the main page by loading the index.html (the file is looking different to yours). Therefore we skipped (commted out) the following coding line...

https://github.com/js-soft/wdi5/blob/ad17fb5fbf5c67b075528f5b9a1eb5435133d847/examples/fe-app/app/incidents/webapp/wdi5-test/Journey.test.js#L6

Maybe this causes the problem ?

The MainPage is working pretty well and tests are executed there but it's not working when the navigation to the detailpage is triggered... there the UI isn't loaded completely and exception is thrown...

vobu commented 2 years ago

here's -hot off the press- the docs for the test lib integration in wdi5: https://js-soft.github.io/wdi5/#/fe-testlib

T1mey commented 2 years ago

@Siolto @vobu

We found the problem ... the code is only working if in manifest.json the table on the main page is having the option

"initialLoad": false

if we change it to true the above mentioned code is failing... Why this have inflence on the object page... we don't know.

Seems to be a bug.

vobu commented 2 years ago

excellent! ain't testing great?! you actually find (and fix!) bugs through it πŸ˜„

T1mey commented 2 years ago

excellent! ain't testing great?! you actually find (and fix!) bugs through it πŸ˜„

@vobu In that case not really ... as we want to have the list loaded on startup. So I think it's a bug in wdi5
Could you add that setting in you test fe application... so your tests will fail as well.

T1mey commented 2 years ago

@vobu

Just add

"initialLoad": true

to

https://github.com/js-soft/wdi5/blob/84e36240af92d69439cafcdc2109ce63acaa909a/examples/fe-app/app/incidents/webapp/manifest.json#L113

in cobination with

Given.onTheMainPage.onTable().iExecuteCreate()
Then.onTheDetailPage.iSeeThisPage().and.iSeeObjectPageInEditMode()
Siolto commented 2 years ago

Hi @T1mey,

cool that you found the issue! But I am not sure if this is a bug in wdi5 itself or on the actual test library. Maybe @ARiesterer could help us here.

T1mey commented 2 years ago

Hi @T1mey,

cool that you found the issue! But I am not sure if this is a bug in wdi5 itself or on the actual test library. Maybe @ARiesterer could help us here.

Maybe keep the ticket open

ARiesterer commented 2 years ago

IΒ΄ll have a look if this test sequence with the initialLoad leads to any issues on our internal OPA tests.

ARiesterer commented 2 years ago

I just tested the following sequence with a RAP application (manifest setting InitialLoad=true):

        opaTest('#0: ListReport: Check creating a new item with setting InitialLoad=true', function(Given, When, Then) {
            When.onTheMainPage.onTable().iExecuteCreate();
            Then.onTheDetailPage.iSeeThisPage().and.iSeeObjectPageInEditMode();
            When.onTheShell.iNavigateBack();
            When.onTheDetailPage.onMessageDialog().iSelectDraftDataLossOption("draftDataLossOptionDiscard");
            When.onTheDetailPage.onMessageDialog().iConfirm();
            Then.onTheMainPage.iSeeThisPage();
       });

got this result:

image

So from test-lib side it should be ok. Pressing the Create button to go to the ObjectPage should be independent from the InitialLoad parameter.

Siolto commented 2 years ago

I just double checked our example (with and without initialLoad: true) and both cases run green for me. There has to be something else that is causing the issue. Maybe some weird side effects with the initialLoad flag.

T1mey commented 2 years ago

I just double checked our example (with and without initialLoad: true) and both cases run green for me. There has to be something else that is causing the issue. Maybe some weird side effects with the initialLoad flag.

I can offer a live demo via teams... no I can't because now it's working with true as well.

What I did in the meantime is to update chromedriver to 103 (I think from 102) ...

T1mey commented 2 years ago

I just double checked our example (with and without initialLoad: true) and both cases run green for me. There has to be something else that is causing the issue. Maybe some weird side effects with the initialLoad flag.

I can offer a live demo via teams... no I can't because now it's working with true as well.

What I did in the meantime is to update chromedriver to 103 (I think from 102) ...

@Siolto @ARiesterer

I did a mistake by adding the initial load property again... so even with 103 it is not working for us and reproducable.

Siolto commented 2 years ago

@T1mey we can schedule a call if you want. Can you send me your email?

github-actions[bot] commented 2 years ago

hey πŸ‘‹ - silence for 30 days 🀐 ... anybody? πŸ˜€

github-actions[bot] commented 2 years ago

closed πŸ“΄ because silencio 🀫 since an additional 14 days after staleness πŸ“