search

nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.78k stars 1.31k forks source link

All methods in async perform() are returning NightwatchAPI instead of actual promise #2872

Open pawlakmaly opened 3 years ago

pawlakmaly commented 3 years ago

Describe the bug

Seems that calling any promise method (both from NW API and custom commands) inside .perform() ends up with getting the Nightwatch API instead of actual promise. I've tested this on NW 1.6.4, 1.7.7 and 1.7.8

Sample test

sampleTest.js

```js // Please add the sample test here import { NightwatchBrowser } from "nightwatch"; module.exports = { after(browser: NightwatchBrowser) { browser.end(); }, "Browser Context": function (browser: NightwatchBrowser) { browser.url("https://www.ecosia.org/").perform(async (browser, done) => { const test = await browser.isVisible({ selector: '[data-test-id="main-nav-menu-link"]', locateStrategy: "css selector", index: 0, }); console.log(test); done(); }); }, }; ```

Run with command

$ nightwatch test/sampleTest.js --your-other-arguments-here

Verbose output

debug.log

```txt [Async Perform Test] Test Suite =============================== - Connecting to localhost on port 9515... Request POST /session { desiredCapabilities: { browserName: 'chrome', chromeOptions: { prefs: { download: [Object], profile: [Object] }, args: [] }, name: 'Async Perform Test' } \ Connecting to localhost on port 9515... / Connecting to localhost on port 9515... Response 200 POST /session (543ms) { sessionId: '427eb7ac04915ac772f1b9bb061213e7', status: 0, value: { acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: 'chrome', chrome: { chromedriverVersion: '91.0.4472.101 (af52a90bf87030dd1523486a1cd3ae25c5d76c9b-refs/branch-heads/4472@{#1462})', userDataDir: 'C:\\Users\\admin\\AppData\\Local\\Temp\\scoped_dir37180_318267638' }, cssSelectorsEnabled: true, databaseEnabled: false, 'goog:chromeOptions': { debuggerAddress: 'localhost:63455' }, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: 'normal', platform: 'Windows', proxy: {}, rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unexpectedAlertBehaviour: 'ignore', version: '92.0.4515.131', webStorageEnabled: true, 'webauthn:extension:largeBlob': true, 'webauthn:virtualAuthenticators': true } i Connected to localhost on port 9515 (591ms). Using: chrome (92.0.4515.131) on Windows platform. Received session with ID: 427eb7ac04915ac772f1b9bb061213e7 → Running command: status ([Function]) Request GET /status Response 200 GET /status (4ms) { value: { build: { version: '91.0.4472.101 (af52a90bf87030dd1523486a1cd3ae25c5d76c9b-refs/branch-heads/4472@{#1462})' }, os: { arch: 'x86_64', name: 'Windows NT', version: '10.0.19043' }, ready: true, message: 'ChromeDriver ready for new sessions.', error: [] } } → Completed command: status ([Function]) (8ms) → Running [before]: → Completed [before]. Running: Browser Context → Running [beforeEach]: → Completed [beforeEach]. → Running command: url ('https://www.ecosia.org/') Request POST /session/427eb7ac04915ac772f1b9bb061213e7/url { url: 'https://www.ecosia.org/' } Response 200 POST /session/427eb7ac04915ac772f1b9bb061213e7/url (1039ms) { sessionId: '427eb7ac04915ac772f1b9bb061213e7', status: 0, value: null } → Completed command: url ('https://www.ecosia.org/') (1042ms) → Running command: perform ([Function]) NightwatchAPI { capabilities: { acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: 'chrome', chrome: { chromedriverVersion: '91.0.4472.101 (af52a90bf87030dd1523486a1cd3ae25c5d76c9b-refs/branch-heads/4472@{#1462})', userDataDir: 'C:\\Users\\admin\\AppData\\Local\\Temp\\scoped_dir37180_318267638' }, cssSelectorsEnabled: true, databaseEnabled: false, 'goog:chromeOptions': { debuggerAddress: 'localhost:63455' }, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: 'normal', platform: 'Windows', proxy: {}, rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: { implicit: 0, pageLoad: 300000, script: 30000 }, unexpectedAlertBehaviour: 'ignore', version: '92.0.4515.131', webStorageEnabled: true, 'webauthn:extension:largeBlob': true, 'webauthn:virtualAuthenticators': true }, currentTest: [Getter], desiredCapabilities: null, sessionId: '427eb7ac04915ac772f1b9bb061213e7', options: { custom_assertions_path: './assertions', custom_commands_path: './commands', default_reporter: 'junit', desiredCapabilities: { browserName: 'chrome', chromeOptions: [Object], name: 'Async Perform Test' }, detailed_output: true, disable_colors: false, disable_error_log: false, dotenv: {}, end_session_on_fail: true, exclude: null, filter: null, globals: { abortOnAssertionFailure: true, abortOnElementLocateError: false, waitForConditionPollInterval: 500, waitForConditionTimeout: 20000, throwOnMultipleElementsReturned: false, suppressWarningsOnMultipleElementsReturned: false, asyncHookTimeout: 60000, unitTestsTimeout: 2000, customReporterCallbackTimeout: 20000, retryAssertionTimeout: 20000, reporter: [Function (anonymous)], default: [Object], integration: [Object], before: [Function: before], beforeEach: [Function: beforeEach], afterEach: [Function: afterEach], isLocal: true }, globals_path: './globals/globalsModule.js', launch_url: '', live_output: true, log_screenshot_data: false, output: true, output_folder: './reports/junit', output_timestamp: false, page_objects_path: './pages', parallel_mode: false, parallel_process_delay: 1000, persist_globals: false, report_command_errors: false, report_prefix: '', screenshots: { enabled: true, filename_format: [Function: filename_format], path: ' on_error: true, on_failure: true }, screenshotsPath: ' selenium: { start_process: false, cli_args: {}, server_path: null, log_path: '', port: undefined, check_process_delay: 500, max_status_poll_tries: 15, status_poll_interval: 200, url: 'http://localhost:9515' }, silent: false, skip_testcases_on_fail: true, skipgroup: '', skiptags: 'firefox', src_folders: null, start_session: true, sync_test_names: true, testWorkersEnabled: false, test_runner: { type: 'default', options: {} }, test_settings: {}, test_workers: false, timestamp_format: '', unit_tests_mode: false, use_xpath: false, webdriver: { start_process: true, cli_args: [], server_path: ', log_path: '', use_legacy_jsonwire: true, check_process_delay: 100, max_status_poll_tries: 10, status_poll_interval: 200, process_create_timeout: 120000, host: 'localhost', port: 9515, ssl: false, proxy: undefined, timeout_options: [Object], default_path_prefix: '', username: undefined, access_key: undefined, url: 'http://localhost:9515' } }, globals: { abortOnAssertionFailure: true, abortOnElementLocateError: false, waitForConditionPollInterval: 500, waitForConditionTimeout: 20000, throwOnMultipleElementsReturned: false, suppressWarningsOnMultipleElementsReturned: false, asyncHookTimeout: 60000, unitTestsTimeout: 2000, customReporterCallbackTimeout: 20000, retryAssertionTimeout: 20000, reporter: [Function (anonymous)], default: { isLocal: true }, integration: { isLocal: false }, before: [Function: before], beforeEach: [Function: beforeEach], afterEach: [Function: afterEach], isLocal: true }, launchUrl: null, launch_url: null, screenshotsPath: '', Keys: { NULL: '', CANCEL: '', HELP: '', BACK_SPACE: '', TAB: '', CLEAR: '', RETURN: '', ENTER: '', SHIFT: '', CONTROL: '', ALT: '', PAUSE: '', ESCAPE: '', SPACE: '', PAGEUP: '', PAGEDOWN: '', END: '', HOME: '', LEFT_ARROW: '', UP_ARROW: '', RIGHT_ARROW: '', DOWN_ARROW: '', ARROW_LEFT: '', ARROW_UP: '', ARROW_RIGHT: '', ARROW_DOWN: '', INSERT: '', DELETE: '', SEMICOLON: '', EQUALS: '', NUMPAD0: '', NUMPAD1: '', NUMPAD2: '', NUMPAD3: '', NUMPAD4: '', NUMPAD5: '', NUMPAD6: '', NUMPAD7: '', NUMPAD8: '', NUMPAD9: '', MULTIPLY: '', ADD: '', SEPARATOR: '', SUBTRACT: '', DECIMAL: '', DIVIDE: '', F1: '', F2: '', F3: '', F4: '', F5: '', F6: '', F7: '', F8: '', F9: '', F10: '', F11: '', F12: '', COMMAND: '', META: '' }, page: { ecosiaPage: [Function: bound pageObjectDefinition], testPage: [Function: bound pageObjectDefinition] }, assert: { fail: [Function: bound assertFn], AssertionError: [Function: bound assertFn], ok: [Function: bound assertFn], equal: [Function: bound assertFn], notEqual: [Function: bound assertFn], deepEqual: [Function: bound assertFn], notDeepEqual: [Function: bound assertFn], deepStrictEqual: [Function: bound assertFn], notDeepStrictEqual: [Function: bound assertFn], strictEqual: [Function: bound assertFn], notStrictEqual: [Function: bound assertFn], throws: [Function: bound assertFn], rejects: [Function: bound assertFn], doesNotThrow: [Function: bound assertFn], doesNotReject: [Function: bound assertFn], ifError: [Function: bound assertFn], match: [Function: bound assertFn], doesNotMatch: [Function: bound assertFn], CallTracker: [Function: bound assertFn], strict: [Function: bound assertFn], attributeContains: [Function: bound queuedCommandFn], attributeEquals: [Function: bound queuedCommandFn], containsText: [Function: bound queuedCommandFn], cssClassNotPresent: [Function: bound queuedCommandFn], cssClassPresent: [Function: bound queuedCommandFn], cssProperty: [Function: bound queuedCommandFn], domPropertyContains: [Function: bound queuedCommandFn], domPropertyEquals: [Function: bound queuedCommandFn], elementNotPresent: [Function: bound queuedCommandFn], elementPresent: [Function: bound queuedCommandFn], hidden: [Function: bound queuedCommandFn], title: [Function: bound queuedCommandFn], titleContains: [Function: bound queuedCommandFn], urlContains: [Function: bound queuedCommandFn], urlEquals: [Function: bound queuedCommandFn], value: [Function: bound queuedCommandFn], valueContains: [Function: bound queuedCommandFn], visible: [Function: bound queuedCommandFn], cookieValueEquals: [Function: bound queuedCommandFn], elementsCount: [Function: bound queuedCommandFn], textMatch: [Function: bound queuedCommandFn] }, verify: { fail: [Function: bound assertFn], AssertionError: [Function: bound assertFn], ok: [Function: bound assertFn], equal: [Function: bound assertFn], notEqual: [Function: bound assertFn], deepEqual: [Function: bound assertFn], notDeepEqual: [Function: bound assertFn], deepStrictEqual: [Function: bound assertFn], notDeepStrictEqual: [Function: bound assertFn], strictEqual: [Function: bound assertFn], notStrictEqual: [Function: bound assertFn], throws: [Function: bound assertFn], rejects: [Function: bound assertFn], doesNotThrow: [Function: bound assertFn], doesNotReject: [Function: bound assertFn], ifError: [Function: bound assertFn], match: [Function: bound assertFn], doesNotMatch: [Function: bound assertFn], CallTracker: [Function: bound assertFn], strict: [Function: bound assertFn], attributeContains: [Function: bound queuedCommandFn], attributeEquals: [Function: bound queuedCommandFn], containsText: [Function: bound queuedCommandFn], cssClassNotPresent: [Function: bound queuedCommandFn], cssClassPresent: [Function: bound queuedCommandFn], cssProperty: [Function: bound queuedCommandFn], domPropertyContains: [Function: bound queuedCommandFn], domPropertyEquals: [Function: bound queuedCommandFn], elementNotPresent: [Function: bound queuedCommandFn], elementPresent: [Function: bound queuedCommandFn], hidden: [Function: bound queuedCommandFn], title: [Function: bound queuedCommandFn], titleContains: [Function: bound queuedCommandFn], urlContains: [Function: bound queuedCommandFn], urlEquals: [Function: bound queuedCommandFn], value: [Function: bound queuedCommandFn], valueContains: [Function: bound queuedCommandFn], visible: [Function: bound queuedCommandFn], cookieValueEquals: [Function: bound queuedCommandFn], elementsCount: [Function: bound queuedCommandFn], textMatch: [Function: bound queuedCommandFn] }, expect: { null: [Function: bound queuedCommandFn], active: [Function: bound queuedCommandFn], attribute: [Function: bound queuedCommandFn], css: [Function: bound queuedCommandFn], enabled: [Function: bound queuedCommandFn], present: [Function: bound queuedCommandFn], property: [Function: bound queuedCommandFn], selected: [Function: bound queuedCommandFn], text: [Function: bound queuedCommandFn], type: [Function: bound queuedCommandFn], value: [Function: bound queuedCommandFn], visible: [Function: bound queuedCommandFn], count: [Function: bound queuedCommandFn], cookie: [Function: bound queuedCommandFn], element: [Function: bound queuedCommandFn], elements: [Function: bound queuedCommandFn], title: [Function: bound queuedCommandFn], url: [Function: bound queuedCommandFn] }, acceptAlert: [Function: queuedCommandFn], back: [Function: queuedCommandFn], contexts: [Function: queuedCommandFn], cookie: [Function: queuedCommandFn], currentContext: [Function: queuedCommandFn], dismissAlert: [Function: queuedCommandFn], doubleClick: [Function: queuedCommandFn], element: [Function: queuedCommandFn], elementActive: [Function: queuedCommandFn], elementIdAttribute: [Function: queuedCommandFn], elementIdClear: [Function: queuedCommandFn], elementIdClick: [Function: queuedCommandFn], elementIdCssProperty: [Function: queuedCommandFn], elementIdDisplayed: [Function: queuedCommandFn], elementIdElement: [Function: queuedCommandFn], elementIdElements: [Function: queuedCommandFn], elementIdEnabled: [Function: queuedCommandFn], elementIdEquals: [Function: queuedCommandFn], elementIdLocation: [Function: queuedCommandFn], elementIdLocationInView: [Function: queuedCommandFn], elementIdName: [Function: queuedCommandFn], elementIdProperty: [Function: queuedCommandFn], elementIdSelected: [Function: queuedCommandFn], elementIdSize: [Function: queuedCommandFn], elementIdText: [Function: queuedCommandFn], elementIdValue: [Function: queuedCommandFn], elements: [Function: queuedCommandFn], execute: [Function: queuedCommandFn], executeAsync: [Function: queuedCommandFn], forward: [Function: queuedCommandFn], frame: [Function: queuedCommandFn], frameParent: [Function: queuedCommandFn], fullscreenWindow: [Function: queuedCommandFn], getAlertText: [Function: queuedCommandFn], getOrientation: [Function: queuedCommandFn], keys: [Function: queuedCommandFn], minimizeWindow: [Function: queuedCommandFn], mouseButtonClick: [Function: queuedCommandFn], mouseButtonDown: [Function: queuedCommandFn], mouseButtonUp: [Function: queuedCommandFn], moveTo: [Function: queuedCommandFn], openNewWindow: [Function: queuedCommandFn], refresh: [Function: queuedCommandFn], screenshot: [Function: queuedCommandFn], session: [Function: queuedCommandFn], sessionLog: [Function: queuedCommandFn], sessionLogTypes: [Function: queuedCommandFn], sessions: [Function: queuedCommandFn], setAlertText: [Function: queuedCommandFn], setContext: [Function: queuedCommandFn], setOrientation: [Function: queuedCommandFn], source: [Function: queuedCommandFn], status: [Function: queuedCommandFn], submit: [Function: queuedCommandFn], timeouts: [Function: queuedCommandFn], timeoutsAsyncScript: [Function: queuedCommandFn], timeoutsImplicitWait: [Function: queuedCommandFn], title: [Function: queuedCommandFn], url: [Function: queuedCommandFn], window: [Function: queuedCommandFn], windowHandle: [Function: queuedCommandFn], windowHandles: [Function: queuedCommandFn], windowMaximize: [Function: queuedCommandFn], windowPosition: [Function: queuedCommandFn], windowRect: [Function: queuedCommandFn], windowSize: [Function: queuedCommandFn], closeWindow: [Function: queuedCommandFn], deleteCookie: [Function: queuedCommandFn], deleteCookies: [Function: queuedCommandFn], end: [Function: queuedCommandFn], getCookie: [Function: queuedCommandFn], getCookies: [Function: queuedCommandFn], getLog: [Function: queuedCommandFn], getLogTypes: [Function: queuedCommandFn], getTitle: [Function: queuedCommandFn], getWindowPosition: [Function: queuedCommandFn], getWindowRect: [Function: queuedCommandFn], getWindowSize: [Function: queuedCommandFn], init: [Function: queuedCommandFn], injectScript: [Function: queuedCommandFn], isLogAvailable: [Function: queuedCommandFn], maximizeWindow: [Function: queuedCommandFn], pause: [Function: queuedCommandFn], perform: [Function: queuedCommandFn], resizeWindow: [Function: queuedCommandFn], saveScreenshot: [Function: queuedCommandFn], setCookie: [Function: queuedCommandFn], setWindowPosition: [Function: queuedCommandFn], setWindowRect: [Function: queuedCommandFn], setWindowSize: [Function: queuedCommandFn], switchWindow: [Function: queuedCommandFn], urlHash: [Function: queuedCommandFn], useCss: [Function: queuedCommandFn], useXpath: [Function: queuedCommandFn], clearValue: [Function: queuedCommandFn], click: [Function: queuedCommandFn], getAttribute: [Function: queuedCommandFn], getCssProperty: [Function: queuedCommandFn], getElementProperty: [Function: queuedCommandFn], getElementSize: [Function: queuedCommandFn], getLocation: [Function: queuedCommandFn], getLocationInView: [Function: queuedCommandFn], getTagName: [Function: queuedCommandFn], getText: [Function: queuedCommandFn], getValue: [Function: queuedCommandFn], isVisible: [Function: queuedCommandFn], moveToElement: [Function: queuedCommandFn], setValue: [Function: queuedCommandFn], sendKeys: [Function: queuedCommandFn], submitForm: [Function: queuedCommandFn], waitForElementNotPresent: [Function: queuedCommandFn], waitForElementNotVisible: [Function: queuedCommandFn], waitForElementPresent: [Function: queuedCommandFn], waitForElementVisible: [Function: queuedCommandFn], changeFrame: [Function: queuedCommandFn], checkElementsNumber: [Function: queuedCommandFn], closeTab: [Function: queuedCommandFn], debug: [Function: queuedCommandFn], getAccessToken: [Function: queuedCommandFn], getBrowserLogs: [Function: queuedCommandFn], getBrowserName: [Function: queuedCommandFn], getElementId: [Function: queuedCommandFn], getJSON: [Function: queuedCommandFn], getParsedBrowserLogs: [Function: queuedCommandFn], getValueAndVerify: [Function: queuedCommandFn], hideElement: [Function: queuedCommandFn], isBrowserLogContainsError: [Function: queuedCommandFn], isChrome: [Function: queuedCommandFn], isEdge: [Function: queuedCommandFn], isElementDisplayed: [Function: queuedCommandFn], isFirefox: [Function: queuedCommandFn], logScreenshot: [Function: queuedCommandFn], mouseOver: [Function: queuedCommandFn], open: [Function: queuedCommandFn], scrollTo: [Function: queuedCommandFn], switchToTab: [Function: queuedCommandFn], waitForTextContains: [Function: queuedCommandFn], waitForTextMatch: [Function: queuedCommandFn], waitForTransitionEnds: [Function: queuedCommandFn] } → Completed command: perform ([Function]) (65ms) → Running command: isVisible ({selector, locateStrategy, index}) Request POST /session/427eb7ac04915ac772f1b9bb061213e7/elements { using: 'css selector', value: '[data-test-id="main-nav-menu-link"]' } Response 200 POST /session/427eb7ac04915ac772f1b9bb061213e7/elements (21ms) { sessionId: '427eb7ac04915ac772f1b9bb061213e7', status: 0, value: [ { ELEMENT: '0.6281572404696572-1' }, { ELEMENT: '0.6281572404696572-2' }, { ELEMENT: '0.6281572404696572-3' }, { ELEMENT: '0.6281572404696572-4' }, { ELEMENT: '0.6281572404696572-5' }, { ELEMENT: '0.6281572404696572-6' }, { ELEMENT: '0.6281572404696572-7' }, { ELEMENT: '0.6281572404696572-8' }, { ELEMENT: '0.6281572404696572-9' }, { ELEMENT: '0.6281572404696572-10' } ] } Request GET /session/427eb7ac04915ac772f1b9bb061213e7/element/0.6281572404696572-1/displayed Response 200 GET /session/427eb7ac04915ac772f1b9bb061213e7/element/0.6281572404696572-1/displayed (40ms) { sessionId: '427eb7ac04915ac772f1b9bb061213e7', status: 0, value: false } → Completed command: isVisible ({selector, locateStrategy, index}) (70ms) → Running [afterEach]: → Completed [afterEach]. No assertions ran. → Running [after]: → Running command: end () → Running command: session ('delete', [Function]) Request DELETE /session/427eb7ac04915ac772f1b9bb061213e7 Response 200 DELETE /session/427eb7ac04915ac772f1b9bb061213e7 (85ms) { sessionId: '427eb7ac04915ac772f1b9bb061213e7', status: 0, value: null } → Completed command: end () (104ms) → Completed command: s ```

Configuration

nightwatch.json

```js { "your": { "config": "here" } } ```

Your Environment

Executable Version
nightwatch --version 164, 1.77, 178
node --version 10.18 and 14.17
Browser driver Version
NAME VERSION
OS Version
NAME VERSION
beatfactor commented 3 years ago

Yes, that's because your testcase is not async. This problem will be fixed in a future version.

Btw: you should maybe consider using describe() as it has a few advantages over this syntax, which I'm assuming is TypeScript? Looking at the example you've posted, I have to say that there is really no benefit of using TS here, especially that you need to use the import statement. Also, in v2, browser will be available as a readonly global which will eliminate the need to worry about using types in your tests and we will rewrite our examples to use describe(). This will continue to work of course, but it will be slightly behind in terms of capabilities.

pawlakmaly commented 3 years ago

@beatfactor yes, I'm using typescript since version 1.3.4 i think, I prefer to have strong typing and static analysis when I work with code :)

Could you tell me a bit more of benefits of using describe syntax ? Honestly I've never use it or dig into it.

In case the the whole test case is async i see other problems with execution of perform in latest version.

beatfactor commented 3 years ago

@pawlakmaly if there are other problems then please let us know so we can look into fixing them. With regards to the describe syntax, here's some things which are easy to do there and which are either too difficult or impractical with the older exports interface:

You can have a look at the complete interface here: https://nightwatchjs.org/guide/using-nightwatch/using-bdd-describe.html

chrisvanrun commented 2 years ago

I have encountered this using a nested waitUtil() call within an async perform. Making the test case asynchronous (on the describe(...) call / it(...) call) made no difference. The weird thing is that it was halfway through the tests that it started to fail.