Broken return from I.executeScript() with NightmareJS?

[jploskonka]

What are you trying to achieve?

I'm trying to return value from inside I.executeScript method with Nightmare helper. What is strange is that with I.executeAsyncScript it works as expected.

I'm not really sure if it's bug or something is going wrong, but with codecept 0.4.X it worked as expected, got broken after upgrading to 1.0.2. I'd like to work on PR to fix it but wanted to make sure that it's really a bug and not me doing something in wrong way.

What do you get instead?

Returned value is undefined

Test suite to reproduce in comparison with executeAsyncScript

const assert = require("assert");

Feature("executeScript and executeAsyncScript");

Scenario("I.executeScript", function* (I) {
  I.amOnPage("/");
  let foo = yield I.executeScript(() => "bar" );

  console.log(foo)
  assert(foo, "bar");
});

Scenario("I.executeAsyncScript without done callback", function* (I) {
  I.amOnPage("/");
  let foo = yield I.executeAsyncScript(() => "bar");

  assert(foo, "bar");
});

Scenario("I.executeAsyncScript with done callback", function* (I) {
  I.amOnPage("/");
  let foo = yield I.executeAsyncScript((done) => done("bar"));

  assert(foo, "bar")
});

Output

CodeceptJS v1.0.2
Using test root "/Users/jakubploskonka/Projects/tmp/whatever"

executeScript and executeAsyncScript --
 I.executeScript
 • I am on page "/"
 • I execute script function()
foo value is:  undefined
 ✖ FAILED in 2767ms

 I.executeAsyncScript without done callback
 • I am on page "/"
 • I execute async script function()
 ✓ OK in 2468ms

 I.executeAsyncScript with done callback
 • I am on page "/"
 • I execute async script function()
 ✓ OK in 2452ms

-- FAILURES:

  1) executeScript and executeAsyncScript: I.executeScript:
     bar

Output with --verbose for failing scenario:

CodeceptJS v1.0.2
Using test root "/Users/jakubploskonka/Projects/tmp/whatever"

executeScript and executeAsyncScript --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   [1] Queued | hook Nightmare._beforeSuite()
 I.executeScript
   Emitted | test.before
   [1] Queued | hook Nightmare._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/")
   [1] Queued | amOnPage: "/"
   Emitted | step.after (I am on page "/")
   [1] Queued | step passed
   [1] Queued | return result
   Emitted | step.before (I execute script function())
   [1] Queued | executeScript: function()
   Emitted | step.after (I execute script function())
   [1] Queued | step passed
   [1] Queued | return result
   [1] Queued | create new promises queue for generator
   Emitted | step.start (I am on page "/")
 • I am on page "/"
 > [URL] https://www.google.de/?gfe_rd=cr&dcr=0&ei=a9GmWZm2Jq6DX9nNvrgN
 > [Code] 200
 > [Headers] {"alt-svc":["quic=\":443\"; ma=2592000; v=\"39,38,37,35\""],"cache-control":["private, max-age=0"],"content-encoding":["gzip"],"content-type":["text/html; charset=UTF-8"],"date":["Wed, 30 Aug 2017 14:53:31 GMT"],"expires":["-1"],"p3p":["CP=\"This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info.\""],"server":["gws"],"set-cookie":["NID=111=Co8fF3OIRuRiRZOVJSVV04W97unfg7-6kRr69Gtqg9jNtskT5XZp6yVc1HgIHGmd5EURznSNg9GIA4rC-CM5A-pz8pNNQFs5P90pOlEXMSOEpE-N1eyIOgaCFOSYxvKD; expires=Thu, 01-Mar-2018 14:53:31 GMT; path=/; domain=.google.de; HttpOnly","CONSENT=WP.264059; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.de"],"status":["200"],"x-frame-options":["SAMEORIGIN"],"x-xss-protection":["1; mode=block"]}
   Emitted | step.passed (I am on page "/")
   Step finished in 2.41 sec
   Emitted | step.start (I execute script function())
 • I execute script function()
   Emitted | step.passed (I execute script function())
   Step finished in 0.011 sec
   [1] Starting <generator> session
foo value is:  undefined
   [1] <generator> Queued | throw error AssertionError [ERR_ASSERTION]: bar
   [1] <generator> Error | AssertionError [ERR_ASSERTION]: bar
   [1] <generator> Starting <teardown> session
   Emitted | test.failed ([object Object])
   [1] <teardown> Queued | hook Nightmare._failed()
   [1] <teardown> Queued | () => done(err)
   [1] <teardown> Stopping recording promises
 > Screenshot is saving to /Users/jakubploskonka/Projects/tmp/whatever/output/I.executeScript.failed.png
 ✖ FAILED in 2983ms

   [2] Starting recording promises
   Emitted | test.after
   [2] Queued | hook Nightmare._after()
   Emitted | suite.after ([object Object])
   [2] Queued | hook Nightmare._afterSuite()

-- FAILURES:

  1) executeScript and executeAsyncScript: I.executeScript:
     bar
  ssertionError [ERR_ASSERTION]: bar
      at Test.<anonymous> (tests/ex_test.js:10:3)
      at Generator.next (<anonymous>)
      at node_modules/codeceptjs/lib/scenario.js:13:24
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)

Details

• CodeceptJS version: 1.0.2
• NodeJS Version: 8.4.0
• Operating System: OSX Sierra
• Nightmare version: 2.10.0
• Nightmare upload: 0.1.1

Config file

{
  "tests": "tests/*_test.js",
  "timeout": 10000,
  "output": "./output",
  "helpers": {
    "Nightmare": {
      "url": "https://google.com",
      "show": true
    }
  },
  "include": {},
  "bootstrap": false,
  "mocha": {},
  "name": "whatever"
}

[joeypedicini92]

Having the exact same problem after upgrading to 1.x

[DavertMik]

Hey, @joeypedicini92 please help us. Could you try the patch from @jploskonka to check if it solves an issue for you.

[joeypedicini92]

@DavertMik I will try and let you know. The move to executeAsyncScript was good enough workaround for me for the time being so I haven't tested this yet

[DavertMik]

@joeypedicini92 please try it as we need to know is PR good enough to get merged

[naveenpgit]

@DavertMik when are you planning to release this? We have exact similar issue if there is plan to push new release with this fix it makes our work lot easier (i do not have to go and update everywhere from executeScript to executeAsyncScript)