ui5-community / wdi5

official UI5 end-to-end test framework for UI5 web-apps. wdi5 = Webdriver.IO + UI5 Test API
https://ui5-community.github.io/wdi5/
Apache License 2.0
102 stars 43 forks source link

browser.mock executed too late #499

Closed mocsa closed 1 year ago

mocsa commented 1 year ago

Describe the bug -> This is not really a bug but a question regarding wdi5 and WebdriverIO internals In my local free-form UI5 test application I have a JSON model in my manifest.json configured like this:

  "sap.app": {
    "dataSources": {
      "myjsonSource": {
          "type": "JSON",
          "uri": "http://192.168.0.10/myjsonapi/"
      }
  }

When I test this application using wdi5, I want to mock the call to myjsonapi using the standard WebdriverIO method documented here, so in mytest.e2e.js I have this:

describe('On the main page', async () => {
    before( 'test something', async () => {

        const mock = await browser.mock('http://192.168.0.10/myjsonapi/');
        mock.respond({
            title: 'This is mocked data',
            completed: false
        })

    });
});

The problem is that the await browser.mock call is executed too late. By the time the test gets to this call, UI5 has already made the call to myjsonapi, therefore, my application will be executed with the production data and not the mocked data.

I'm not sure if it matters, but in wdio.conf.js I don't have a url parameter in the wdi5 object and baseUrl is set to http://192.168.0.10/myui5app/webapp/ where I have my index.html file.

Expected behavior I would like to know where shall I put my call to await browser.mock so that it is executed before UI5 calls myjsonapi. This would have the effect that when UI5 actually makes the call WebdriverIO would supply the mocked response to my application.

I'm not a WebdriverIO expert but I guess the question boils down to how wdi5 starts the UI5 application. This is not explained in the wdi5 docs. It seems to me that in WebdriverIO you have to specifically call browser.url() to start your web application but wdi5 seems to navigate to baseUrl automatically.

Runtime Env (please complete the following information):

vobu commented 1 year ago

hi, what you're experiencing is most likely due to the preload behavior of models in UI5. We're only able to inject the UI5 Test API at wdi5 runtime after the UI5 bootstrap has happened - eventually including the call to preload any models. Have you tried setting preload: false in the manifest.json's model definition (https://ui5.sap.com/sdk/#/topic/26ba6a5c1e5c417f8b21cce1411dba2c)?

mocsa commented 1 year ago

I previously did not have the preload setting in my manifest.json so I added it like this:

  "sap.ui5": {
    "models": {
      "myjsonModel": {
        "type": "sap.ui.model.json.JSONModel",
        "dataSource": "myjsonSource",
        "preload": false
      }
    }
  }

but the model is still preloaded.

My application starts with a basic page which does not use the model. I also checked and I don't have any JS code which uses the model during application startup.

The strange thing is that the documentation you referenced says that the default is actually preload: false, so preload should not have happened even before I added this setting.

In Chrome Dev Tools I took a screenshot of the "Initiator" tab of the network request to the JSON source. It seems that the model is preloaded without touching my code at all.

Do you have any other hints how to make preload: false work? Alternatively, shall I maybe try using skipInjectUI5OnStart? So far I tried to avoid skipInjectUI5OnStart because I hoped there is a cleaner solution (since my app is pure UI5), but maybe I have no other choice. I have also seen that at the end of wdio.conf.js there are many WebdriverIO hooks. I'm not familiar with them but can you perhaps tell if any hooks are called before Test API injection and therefore can be used to mock the model preload?

image

dominikfeininger commented 1 year ago

This runs successful.

I guess since the model is configured in manifest.json the init method runs quite before the test execution -> use the before hook to create your mock data and/ or use the late inject.

in wdio-ui5-late.conf.ts

baseConfig.before = async () => {
    const mock = await browser.mock("http://localhost:8080/V2/Northwind/Northwind.svc/Customers('TRAIH')")
    mock.respond({
        d: {
            __metadata: {
                uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')",
                type: "NorthwindModel.Customer"
            },
            CustomerID: "TRAIH",
            CompanyName: "Trail's Head Gourmet Provisioners",
            ContactName: "This is mocked data",
            ContactTitle: "Sales Associate",
            Address: "722 DaVinci Blvd.",
            City: "Kirkland",
            Region: "WA",
            PostalCode: "98034",
            Country: "USA",
            Phone: "(206) 555-8257",
            Fax: "(206) 555-2174",
            Orders: {
                __deferred: {
                    uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')/Orders"
                }
            },
            CustomerDemographics: {
                __deferred: {
                    uri: "https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TRAIH')/CustomerDemographics"
                }
            }
        }
    })
}

in ui5-late.test.js

    it.only("wdi5 should work regularly with mocked data", async () => {
        // native wdio functionality - navigates to the wdi5 github page
        await browser.$("#user-content-wdi5-").waitForDisplayed()
        // open local app
        await browser.url("http://localhost:8080/index.html")
        // do the late injection
        await wdi5.injectUI5()

        const inputText: wdi5Selector = {
            selector: {
                id: "mainUserInput",
                viewName: "test.Sample.tsapp.view.Main"
            }
        }

        const webcomponentValue = await (browser.asControl(inputText) as unknown as Input)

        expect(await webcomponentValue.getValue()).toEqual("This is mocked data")
    })

Happy to if you can work out and contribute a best practice recipe using the browser.mock funtionality.

mocsa commented 1 year ago

With your suggestions I was able to make mocking work. However, I find this method quite complicated.

So far, my idea was that to test one application I will create one high-level config-file. Then I will test different behaviors in different describe-it tests, using the same config-file. But this seems to be impossible with mocked data.

I admit, my hope with creating this issue was that you might have insider information about the initialization process of UI5 (or have colleagues whom you can ask) and you can come up with a finely-targeted point of time where the mocked data can be injected, without messing up the test by introducing per-test config-files, late-injecting UI5, waiting for an #id on an external page (so now the test depends on the correct #id), etc.

Siolto commented 1 year ago

Hi @mocsa,

why don't you use the help of the mockserver? With the V2 MockServer you also have the possibility to intercept simple REST calls and return data as you like.

mocsa commented 1 year ago

I haven't thought of that. I tried to remain within the realms of WebdriverIO and wdi5, and not adding another dependency unless absolutely necessary. Since WebdriverIO already has mocking capabilities, I tried to make that work. Also, I have zero experience with MockServer.

I guess, you mean sap.ui.core.util.MockServer. I briefly checked the documentation but I could only find examples for mocking OData, however I'm using JSON.

Also, is it easy in MockServer to have different mocked data for different tests?

Could you perhaps give a brief overview of what settings I need to put into what files to make your suggestion work?

Siolto commented 1 year ago

for most of your questions I would need a look at your setup and use-case to give you concrete guidance here.

For me it is also unclear if you want to write integration tests or real e2e tests? Maybe you can have a look at this blog post

We are soon able to offer sponsored support where we then could explain you everything in more detail but for now I would close this issue.

mocsa commented 1 year ago

Sorry, Simon, but I'm not getting much help from your comments. Why have you closed this issue? It is not solved. I also don't know what you mean by "sponsored support". Is that paid support? It sounds a bit out-of-place in an open source project. I'm just helping this project in my free time for no money. But sorry if I misunderstood you.

Anyways...

After implementing @dominikfeininger's suggestion I tried to bring it closer to my original expectation and I partly succeeded. I'm putting it here for the benefit of others.

I wanted to have different mock responses for each of my tests, so I changed @dominikfeininger's above code by moving the setup-code of the mock response from the wdio.conf.js file to the ui5.test.js file and it actually worked. Basically, I had to implement the late injection of the wdi5 framework and this allowed me to set up the mock response before UI5 loaded the data of my model configured in manifest.json.

I did the following:

  1. In my wdio.conf.js I changed baseUrl from the index page of my UI5 app to an arbitrary other webpage which does not start my UI5 application.

  2. I added skipInjectUI5OnStart=true in wdio.conf.js.

  3. In my test.e2e.js file I did something like this:

const { default: _ui5Service } = require("wdio-ui5-service")
const ui5Service = new _ui5Service()
/* Other require statements */

describe('On the subpage', async () => {
    before( 'when something is done', async () => {
        const mock = await browser.mock('http://192.168.0.10/myjsonapi/');
        mock.respond({
            title: 'This is mocked data',
            completed: false
        })
    // The following line should wait for an element on the page in your baseUrl
    await browser.$("#wrapper").waitForDisplayed()
    // open your UI5 app
    // this will load the JSON data for your model
    // but by now the mock response has been set up, so the model will be
    // filled with mock data
    await browser.url("http://192.168.0.10/myui5app/webapp/index.html")
    // late inject the wdi5 framework into your app
    await ui5Service.injectUI5()    

    });
    it('ordering of table should work', async () => {

        // Do your test here, interact with your UI5 app

        // assert your test result
        expect(value1 <= value2).toBeTruthy

    });
});