search

xolvio / meteor-rtd-example-project

A template project to use for creating a meteor app with unit and webdriver testing
91 stars 27 forks source link

Code coverage broken: webdriver.By.className('metric') is undefined #14

Closed wenzowski closed 11 years ago

wenzowski commented 11 years ago

Just checked this out to try and get a basic idea of how RTD works.

Test suite is completing successfully, but can't get coverage working.

$ grunt --debug
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/Gruntfile.js
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/Gruntfile.js
Launching Selenium-server on port 4444
Launching Karma listener on port 9876
Launching Karma runner on port 9100
Launching Meteor on port 3000
Launching Mirror on port 8000
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-contrib-watch/tasks/watch.js
Waiting...[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee

- - - Running unit tests - - -
...........
PhantomJS 1.9 (Mac): Executed 11 of 11 SUCCESS (0.139 secs / 0.006 secs)
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
- - - Running acceptance tests - - -
.

Finished in 9.144 seconds
1 test, 4 assertions, 0 failures, 0 skipped

[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/Gruntfile.js
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
[D] Task source: /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/grunt-bg-shell/tasks/bg-shell.coffee
- - - Running coverage tests - - -

>> { [TypeError: Cannot call method 'getText' of undefined]
>>   stack: 'TypeError: Cannot call method 'getText' of undefined
    at /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/lib/final.spec.js:107:28
    at /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/goog/base.js:1112:15
    at webdriver.promise.ControlFlow.runInNewFrame_ (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:1431:20)
    at notify (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:315:12)
    at notifyAll (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:284:7)
    at fulfill (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:389:7)
    at Object.webdriver.promise.asap (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:665:5)
    at webdriver.promise.ControlFlow.runInNewFrame_ (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:1442:25)
    at notify (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:315:12)
    at notifyAll (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:284:7)
==== async task ====
WebDriver.findElements(By.className("metric"))
    at webdriver.WebDriver.schedule (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:246:15)
    at webdriver.WebDriver.findElements (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:798:17)
    at /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/lib/final.spec.js:105:20
    at /Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/goog/base.js:1112:15
    at webdriver.promise.ControlFlow.runInNewFrame_ (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:1431:20)
    at notify (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:315:12)
    at notifyAll (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:284:7)
    at fulfill (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:389:7)
    at Object.webdriver.promise.asap (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/node_modules/selenium-webdriver/lib/webdriver/promise.js:665:5)',
  webdriver_promise_error_: true }
Error in acceptance tests

Completed in 16.185s at Tue Jul 02 2013 14:40:09 GMT-0400 (EDT) - Waiting...

...and in the browser...

TypeError: Cannot read property 'start' of undefined
    at tmp (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/lib/istanbul-middleware-port/.npm/node_modules/istanbul/lib/object-utils.js:58:44)
    at Array.forEach (native)
    at Object.addDerivedInfoForFile (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/lib/istanbul-middleware-port/.npm/node_modules/istanbul/lib/object-utils.js:57:37)
    at Object.Collector.fileCoverageFor (/Volumes/Wenzowski-HD/Users/alex/code/github/real-time-development-with-meteor/test/rtd/lib/istanbul-middleware-port/.npm/node_modules/istanbul/lib/collector.js:93:15)
    at app/packages/istanbul-middleware-port/core.js:143:86
    at Array.forEach (native)
    at getTreeSummary (app/packages/istanbul-middleware-port/core.js:142:23)
    at Object.core.render (app/packages/istanbul-middleware-port/core.js:186:19)
    at app.get.origUrl (app/packages/istanbul-middleware-port/handlers.js:31:14)
    at Object.<anonymous> (app/packages/istanbul-middleware-port/express-shim.js:16:13)

Same trace with node v0.10.5, v0.10.10, v0.11.2 via nvm on Mac OS 10.8.4.

Am I missing something silly?

xolvio commented 11 years ago

Hi. I just updated the codebase to fix some issues. Would you mind updating and trying again?

wenzowski commented 11 years ago

Pulled, updated all submodules, same error.

If it helps diagnosing this, I had to open http://localhost:4444/wd/hub and manually add a chrome session.

Until this is resolved, is there a way to disable code coverage from config?

rdickert commented 11 years ago

Hi there, same issue for me, just test driving the stock project on osx. I had to run the brew install phantomjs (or attach a browser at localhost:9876) to solve the problem of not having a browser connected. Now the unit & acceptance tests run, but the coverage tests fail similarly to the above. Here's the output on the console:

- - - Running unit tests - - -
...........
PhantomJS 1.9 (Mac): Executed 11 of 11 SUCCESS (0.064 secs / 0.005 secs)
- - - Running acceptance tests - - -
.

Finished in 7.579 seconds
1 test, 4 assertions, 0 failures, 0 skipped

- - - Running coverage tests - - -

>> { [TypeError: Cannot call method 'getText' of undefined] webdriver_promise_error_: true }
Fatal error: { [TypeError: Cannot call method 'getText' of undefined] webdriver_promise_error_: true }

OSX 10.8.4 with node 0.10.5, Meteor 0.6.4. Let me know if I can provide any additional info.

xolvio commented 11 years ago

I think this is relates to this issue.

You can disable coverage by copying /test/rtd/rtd.conf.js to /test/rtd.conf.js and commenting out the coverage checks to have this:

    startupTasks: [
        'bgShell:killAll',
        'downloadAndOrStartSelenium',
        'bgShell:synchronizeMirrorApp',
        // 'bgShell:instrumentCode',
        'bgShell:startMirrorApp',
        'bgShell:startKarma',
        'bgShell:startApp',
        'outputPorts',
        'watch'
    ],
    watchTasks: [
        'bgShell:karmaRun',
        'bgShell:synchronizeMirrorApp',
        // 'bgShell:instrumentCode',
        'bgShell:runTests',
        /// 'postLatestUnitCoverage',
        'bgShell:killReports',
        // 'bgShell:runCoverageCheck'
    ],
xolvio commented 11 years ago

Could one of you please zip up the entire directory and send it to me so I can diagnose it using something like fileshare or google drive etc?

Thanks

wenzowski commented 11 years ago

Thanks! Almost had it, I'd missed the bgShell:instrumentCode in startupTasks.

Launching Selenium-server on port 4444
Launching Karma listener on port 9876
Launching Karma runner on port 9100
Launching Meteor on port 3000
Launching Mirror on port 8000
Waiting...

Then manually start a webdriver session from http://localhost:4444/wd/hub and

- - - Running unit tests - - -
...........
PhantomJS 1.9 (Mac): Executed 11 of 11 SUCCESS (0.068 secs / 0.006 secs)
- - - Running acceptance tests - - -
.

Finished in 6.444 seconds
1 test, 4 assertions, 0 failures, 0 skipped

Done, without errors.

Completed in 12.303s at Wed Jul 03 2013 19:31:32 GMT-0400 (EDT) - Waiting...
wenzowski commented 11 years ago

tar'd entire working directory after cloning, performing setup indicated in readme, and running grunt. Thanks for offering to take a look through.

xolvio commented 11 years ago

Great. Seems odd that you have start the webdriver session manually, you shouldn't have to.

I've just updated the config file to hide away the details of startup / watch tasks. You can now set coverage to on/off

xolvio commented 11 years ago

Thanks for sending that through. Running the directory you sent me as it is worked straight away on my machine, without having to manually start selenium and the coverage checks worked, which tells me this environmental problem, and of course you can't tar your machine and send it to me!

What version of node do you have there? I'm running v0.10.12 on OSX 10.8.4

wenzowski commented 11 years ago

I've been developing in v0.11.2, but tried v0.10.10 and v0.10.5. Will try v0.10.12.

Not sure what's wacky about my environment. I had to install phantom via homebrew.

$ which -a phantomjs
/usr/local/bin/phantomjs
/Users/alex/.nvm/v0.10.12/bin/phantomjs

While the npm version does install successfully it doesn't work as expected. When I remove homebrew phantom, and use npm phantom instead:

$ grunt
Launching Selenium-server on port 4444
Launching Karma listener on port 9876
Launching Karma runner on port 9100
Launching Meteor on port 3000
Launching Mirror on port 8000
Waiting...
- - - Running unit tests - - -
>> Error: Command failed:
Fatal error: Command failed:

Completed in 1.998s at Wed Jul 03 2013 22:22:03 GMT-0400 (EDT) - Waiting...
wenzowski commented 11 years ago

that last one is just cryptic with debug turned off.

- - - Running unit tests - - -
No captured browser, open http://localhost:9876/
>> Error: Command failed:
Fatal error: Command failed:

the only difference with npm phantom instead of homebrew phantom is that Karma also needs to be started manually as well.

xolvio commented 11 years ago

try to change phantom to chrome in karma.conf.js (you can also copy this to your /test dir and it will override the one in /test/rtd)

wenzowski commented 11 years ago

That did it! Thanks for the help.

Now to figure out why phantomjs is broken...

xolvio commented 11 years ago

Thanks for letting me know. FYI, I'm using Phantom 1.9.0

lc3t35 commented 11 years ago

Works fine with phantomjs 1.9.0 instead of 1.9.1-1 !