UI5 dependencies are not loaded on this page

[the-docta]

Hi guys, I am relatively new to uiveri5, so maybe I'm just missing something...

I have a SAPUI5 app ( yes, it's a ui5 app, cf. #249 )

using the page concept with When.onThePage... - Then.onThePage...

all tests are working fine, with one exception: if a test clicks a button, that triggers a navigation (Router.navTo in ui5), then the next (and all following) element(by.control(...)) will throw "UI5 dependencies are not loaded on this page" error. (full error below)

observations so far:

• navigation from FLP to app (to default route of the app) => works fine
• navigation from default route in app to detail route in same app => works fine
• navigation from detail route to detail route with different parameters =>
  • if run with chrome driver, I can visually see that the whole page is reloaded rather than just replacing the binding), and every by.control after that fails with "UI5 dependencies are not loaded on this page" (full error below)
  • if run with chromeHeadless driver, I have no visual control of whether the whole page has reloaded, but get same error

now, following a suggestion in #249 I added a browser.loadUI5Dependencies before running next line of test, now sometimes the test fails with same above error, sometimes not, when using chrome driver and still always fails when using chromeHeadless

When.onTheAppPage.iClickButtonByIdInDetailView("idDetailEditButton")
 // when navigating, ui5 libs are not always loaded already (timing problem??), so force load them
browser.loadUI5Dependencies().then(() => {
    Then.onTheAppPage.iSeeItemInDetailView(...)
})

all in all this feels like some sort of timing problem. tried increasing timing values in the conf.js

using sapui5 1.84.4 if that#s of relevance, uiveri5 1.46.2 (most current as of today)

full error:

INFO: Expectation FAILED: Failed: javascript error: UI5 dependencies are not loaded on this pageError: UI5 dependencies are not loaded on this page
    at findByControl (eval at executeScript (:480:16), <anonymous>:5:13)
    at eval (eval at executeScript (:480:16), <anonymous>:39:6)
    at executeScript (<anonymous>:482:30)
    at <anonymous>:487:24
    at callFunction (<anonymous>:450:22)
    at <anonymous>:464:23
    at <anonymous>:465:3
  (Session info: chrome=88.0.4324.182)
  (Driver info: chromedriver=88.0.4324.96 (68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324@{#1784}),platform=Mac OS X 11.2.1 x86_64)

[maximnaidenov]

Hi, some background how uiveri5 works. When you do a manual browser.go() or have the declarative authentication, we do the auth but we also inject our instrumentation by calling loadUIDependecies() internally. When the app does page reload or navigates to another page, this instrumentation is not loaded in the new page and you get this UI5 dependecies error on first synchronization e.g. first action or assertion. Cross-app testing is tricky as we can't detect this page reload and inject automatically and so you need to do it manually. But before that, you need to make sure the new page has fully loaded. You can do it with browser.driver.wait() for some condition, like some visible element on the new page. And just then call loadUI5Instrumentation() and then it will run reliably. We have some hints here: https://github.com/SAP/ui5-uiveri5/blob/master/docs/usage/browser.md

[the-docta]

Thanks for the quick response, I will take a look at the driver.wait and the browser page.

The thing is, the navigation between pages is only a change in hash. the problematic case doesn't even go from one app to a different one. and under normal conditions (human presses that button), the browser does not reload the page at all, if the human presses that button in example, app-internal navigation is triggered, and after navigation, some code in the app changes the binding of the view, which was already rendered. so really only the data is exchanged.

Thus, I was wondering, why the uiveri5-driven browser did a full page-reload at all

(if we could figure out, why, and could prevent it, we wouldnt need to reinject the ui5 dependences)

[maximnaidenov]

UIVeri5 does NOT DO page reloads, it is the app doing it. Do not forget that in normal operation, the browser caches the page heavily and IMHO, you just miss to see the full reload. But with selenium, the browser is started with new profile and some features are disabled, like caching. So you see such stuff visually.

[the-docta]

Well, I am not saying, UIVeri5 does the reload, maybe it's the browser driver or something.

When the button is clicked, all that happens is changing the routing (the part of URL after the hash).

in normal operation, an event handler of the app is notified about that change, requests new data and binds already visible controls to that data. This is also clearly visible in the network trace.

if the same button is clicked in a uiveri5 test, the whole page (including the HTML page) is loaded. What I did:

• run uiveri5 --debug using chrome browser
• put a breakpoint on the line, which does the element(...).click()
• when the breakpoint was hit, I opened the dev tools in the automated browser window and opened the network trace
• just to see I understand, what's being checked, a window.uiveri5 in the console of the automated browser reveals an object
• after that I continued the test scrpit so it did the .click()
• After that I can clearly see in the automated browser that it reloads the whole page including the HTML file, all scripts, styles, everything

so the page really reloads. It was not an impression I had.

[the-docta]

But with selenium, the browser is started with new profile and some features are disabled, like caching.

the browser might start with an empty cache, but I also clearly see in the network trace of the automated browser that it got most of the files from disk cache and some even from memory cache, when the page was reloaded at the specified point in the test I described in my previous comment.

[maximnaidenov]

I can assure you that UIVeri5 does not do browser reload (unless explicitly instructed with code). You can confirm this yourself, search for reloads in the source code. I also have not seen webdriverjs/selenium doing page reload. But I have seen many apps, especially in FLP/app-to-app navigation scenarios doing reloads. Actually this is the reason we exposed this loadUI5Dependencies() method, it was requested by several apps over the years. Can you please try the following: execute EXACTLY the same scenario in incognito view and check in the network logs that there is no page reload.

[the-docta]

Just checked: no reload in incognito mode in normal operation

[the-docta]

Well, I found the difference (don't know WHY it makes the difference, though), maybe we can find out with common effort:

• the data service (and only the data service) is protected by basic auth in the test environment
• So I used auth: {basic: {user:..., pass:...}} in the configuration file.

when I ran the example in incognito mode, I realised that the test script probably sends the Authorization header with EVERY request. So I removed the auth block from configuration file and placed a breakpoint before the first test, so I had enough time to manually enter the basic auth credentials in the automated browser, when it asked for it.

Then I continued the test end everything ran smoothly without page reloads

[maximnaidenov]

Hi, this whole thing sounds as there is something wrong with the app (like incompatible backend auth e.g. using a bearer token/certificate SSO) or most likely the system config (basic auth in url is strongly discouraged even on https connection and as as I know, not supported on recent SAP backends) UIVeri5 is an E2E testing framework that is automating user interacting. Users click on buttons and enter in fields, they do not add auth headers. So UIVeri5 does not have support for changing headers. What the basic auth authenticator does is so prepend the baseUrl with credentials. Then the server that serves the app home page is supposed to parse credentials, do the authentication and respond with a cookie. Then any further requests made from the app to the same domain (and the backend exposed as rest endpoint) will have the cookie and the app will work fine. But as mentioned, this depends on the server serving the app home page. This scheme was working fine in NetWeaver few years back but is not supported in recent FLP/S4Cloud versions. Instead OAuth2 with or without SSO is used by default (SAP IDM is the only possible scheme on SAP CP). So please have a look at your server config and fix it. But the best is to change to OAuth2 auth that we support as "sapcloud-form" authConfig..

[the-docta]

Hi, this whole thing sounds as there is something wrong with the app

please stop blaming the app in every comment.

What the basic auth authenticator does is so prepend the baseUrl with credentials

Well, that is the problem. SAPUI5 will rewrite the URL to not contain the user and password anymore. And when it does, the browser reloads the page, because the URL changed. not only the part behind the hash.

https://sapui5.hana.ondemand.com/1.84.4/resources/sap/ui/thirdparty/hasher-dbg.js => search for Method replaceHash

that is at least incompatible with basic auth the way it is implemented in Uiveri5, because it replaces the whole thing:

if a url http://user:pass@host/path#hash is opened, the url returned by window.location will not contain the user and the password parts (for security reasons) => so the new URL placed by SAPUI5 will not contain user and password anymore. Thus, different url, thus the page reload.

[the-docta]

PS: ui5-internally, replaceHash method is not used always when navigating inside the app => only when certain parameters are used (replace history). That explains why some navigations work as expected and some don't.

[maximnaidenov]

You can consider basic auth scheme as deprecated in UIVeri5 and stop using it. It works for the intended usecase with NetWeaver and this is everything we support.

[jiangxin0503]

So how it works now or solution for this, because we are also facing this issue?

[the-docta]

As @maximnaidenov mentioned, UIveri5 considers basic auth as deprecated.

I developed a small extension, which basically does the following:

1. go to test url with basic auth privided by uiveri5 (uiveri5 will internally use http://user:pass@host format)
2. after that, go to an url without basic auth
3. browser remembers from step 1 and sends authorizsation headers to service. no redirects on navigation happen any more.

feel free to use, but note that it is a "works for me" solution, that I made in a generic manner, but has not been heavily tested outside of our own projects

File customBasicAuth.js

"use strict"

const basicAuth = require('@ui5/uiveri5/src/authenticator/basicUrlAuthenticator')

let done = false
function CustomBasicUrlAuthenticator(config, instanceConfig, logger) {
    if (!done) {
        const basicAuthInstance = basicAuth(config, instanceConfig, logger)
        basicAuthInstance.get(instanceConfig.url)
        done = true
    }
}

CustomBasicUrlAuthenticator.prototype.get = function (url) {
    return browser.driver.get(url)
}

module.exports = function (config, logger) {
    return new CustomBasicUrlAuthenticator(config, logger);
};

usage in conf.js:

const customBasicAuth = require.resolve("./extensions/auth/customBasicAuth")
exports.config = {
    ...
    baseUrl: "http://localhost:8080/index.thml",
    authConfigs: {
        customBasicAuth: {
            name: customBasicAuth
        }
    },
    auth: {
        customBasicAuth: {
            url: "http://localhost:8080/path/to/service/that/actually/requires/basic/auth",
            user: "username",
            pass: "password123"
        }
    }
}

Pls let me know if it works well for you. If it dies, I could provide a PR for uiveri5 and/or provide it as separate extension.

[maximnaidenov]

@the-docta But this is exactly how "our" basic auth is working, we append the user:pass only in the browser.driver.get(). Or you mean that subsequent browser.go() should not use the auth again ? If it is only that, we could merge a fast fix behind a param.

[maximnaidenov]

@jiangxin0503 Please describe your problem in a separate issue, "UI5 dependencies not loaded" is pretty generic problem that happens when the loaded page is not UI5. This could be an login page, error page or simply empty page.

[the-docta]

Hi Maxim, we discussed it in great detail above. This is what happens with plain valinna basicauth provided by uiveri5, you can replay this in your browser without uiveri5

1. go to baseUrl, extended by user:pass in URL, e.g. http://user:pass@localhost:8080/index.html
2. browser will send Authorizarion header to all subsequent requests, which responded with 401
3. although, the user:pass part is NOT VISIBLE in url of the browser (for security reasons), and also NOT CONTAINED in window.document.location (also for security reasons), it is still there.
4. on navigation, replaceHash method in sapui5 overwrites whole window.document.location (see my above comments for the exact place of method), which IS A URL CHANGE for the browser, so the browser issues a full refresh of the page, which causes the problem
5. After that, the URL is http://localhost:8080/index.html (without user/password)
6. After that, no refresh happens anymore

so my "custom basic auth handler" does the following

1. go to a service URL, which really requires the authentication with standard-basic-auth of uiveri5, basically calling http://user:pass@localhost:8080/path/to/service
2. After that, UIveri5 goes without adding user/pass to baseUrl, e.g. http://localhost:8080/index.html
3. now, browser still adds the Authorization haeders to all subsequent service calls, as the user is remembered by the browser to be logged in.
4. now, on navigation, saui5 still overwrites the whole URL, but the url remains really the same for the browser and the browser does not initiate a full refresh of the page.

Note, that step4 of both above scenarios is only a problem, when navigtion is executed which should not remain in history. i.e. Router.navTo(..., /* bReplace= */ true , ...)

[maximnaidenov]

@the-docta Now finally this all makes sense to me.Sorry that I was not able to comprehend it initially but basic auth is strongly discouraged in internal SAP systems. All internal systems were reconfigured some time ago and now all use IDM e,g, oauth2 flow e.g. formAuthenticator. I will glad to merge a fix in the basicUrlAuthentticator to do this second browser.driver.go with plain url behind a flag. Or we could implement it but could not really test it.

[YaswanthNakkina]

Recently I faced this issue where Ui5 dependencies are not loaded.

Then I tried with the following approach browser.driver.get('url',{auth:{}}); browser.loadUI5Dependencies();

It worked for me 😊