Assertions do not lead to error

[FrankVisuals]

Describe the bug

I had this bug open https://github.com/ui5-community/wdi5/issues/501 a few month ago, but it got closed due to inactivity. Now I am facing this issue again - in a fresh project.

I have an expect that is very obviously wrong. I followed to guide to init a new project using npm init wdi5@latest -- --ts and this test was generated.

I removed the skip and executed it - it works. However, I don't have the namespace ui5.typescript.helloworld.view.App in my app. It should fail.

I added the line expect(app).toBeDisplayedInViewport(); in order to try to force a test failure, but as you can see, the error log is coming up, but the test succeeds. This seems to be the same issue I described in #501 - only this time, I have a fresh project running.

To Reproduce Steps to reproduce the behavior:

1. Setup fresh project using npm init wdi5@latest -- --ts
2. Remove the skip in the added test
3. Change to correct baseUrl in wdio.conf.ts
4. Run tests using npm run wdi5

UPDATE:

I've just cloned this repo and adapted the Basic.test.ts in the ts-example app. I rewrote the test to

    it("show example", async () => {
        const allButtonsSelector: wdi5Selector = {
            selector: {
                controlType: "whatever",
                viewName: "notexistingwhatever"
            }
        }

        const button = await browser.asControl<Button>(allButtonsSelector)
        expect(button).toBeDefined()
        expect(button).toBeDisplayed()
    })

This should clearly fail, but still leads to success.

Expected behavior The test should fail if the control does not exist.

Logs/Console Output if applicable, please copypasta code-fenced log output, e.g.

Execution of 1 workers started at 2024-07-04T13:55:33.354Z

[0-0] RUNNING in chrome - file:///webapp/test/e2e/sample.test.ts
[0-0] [wdi5] hello world!
[0-0] [wdi5] call of _getControl() failed because of: Error: No DOM element found using the control selector {"controlType":"sap.m.App","viewName":"ui5.typescript.helloworld.view.App"}
[0-0] [wdi5] error retrieving control: ui5typescript.helloworld.view.Appsap.m.App
[0-0] PASSED in chrome - file:///webapp/test/e2e/sample.test.ts

 "spec" Reporter:
------------------------------------------------------------------
[chrome 126.0.6478.127 mac #0-0] Running: chrome (v126.0.6478.127) on mac
[chrome 126.0.6478.127 mac #0-0] Session ID: 9655ccd05439ca2333df6e1700e6ce04
[chrome 126.0.6478.127 mac #0-0]
[chrome 126.0.6478.127 mac #0-0] » /webapp/test/e2e/sample.test.ts
[chrome 126.0.6478.127 mac #0-0] samples
[chrome 126.0.6478.127 mac #0-0]    ✓ should log
[chrome 126.0.6478.127 mac #0-0]    ✓ should retrieve a UI5 control
[chrome 126.0.6478.127 mac #0-0]
[chrome 126.0.6478.127 mac #0-0] 2 passing (8.3s)

Runtime Env (please complete the following information):

• "@wdio/cli": "^8.39.1"
• "@wdio/local-runner": "^8.39.1"
• "@wdio/mocha-framework": "^8.39.0"
• "@wdio/spec-reporter": "^8.39.0"
• "wdio-ui5-service": "^2.0.9"
• UI5 1.120.11
• wdio 8.39.1
• node v18.16.1
• OS: MacOs
• Chrome 126.0.6478.62

Additional context

Test was executed non-headless and locally, on my machine

[github-actions[bot]]

hey 👋 - silence for 30 days 🤐 ... anybody? 😀

[vobu]

we have some ongoing discussion how to best handle error cases. This definitely falls into that and should err out hard. Will take a look as soon as time permits.

[github-actions[bot]]

hey 👋 - silence for 30 days 🤐 ... anybody? 😀

[FrankVisuals]

Is the discussion done offline or can it be followed in github?

[Siolto]

Hi @FrankVisuals,

this one of the topics we still have to tackle. In the meantime:

When you just want to check that a control was found you can do something like this:

const app = await browser.asControl({...})

expect(app.isInitialized()).toBeTruthy()

This should really throw an error when the control was not found

[github-actions[bot]]

hey 👋 - silence for 30 days 🤐 ... anybody? 😀

[github-actions[bot]]

closed 📴 because silencio 🤫 since an additional 14 days after staleness 📠

[FrankVisuals]

Sadly, this got auto clsoed, is there any update on this?

[github-actions[bot]]

hey 👋 - silence for 30 days 🤐 ... anybody? 😀