search

holidayextras / jsonapi-client

Easily consume a json:api service in Javascript.
MIT License
68 stars 19 forks source link

Need any help getting the tests passing in master? #43

Open gabesmed opened 8 years ago

gabesmed commented 8 years ago

Hey @theninj4, we love this JSON API client and would love to make improvements to it, but our hands are tied because the tests are failing in the master branch.

Confirmed they are failing in master: https://github.com/holidayextras/jsonapi-client/pull/42

Additionally we've been unable to get the karma tests running locally, either on a recent Macbook Pro, or through an ubuntu vagrant instance. How do you recommend running the test suite locally?

Thanks a ton for your time! We'd love to pitch in but without even being able to run tests locally, it's hard to get started.

theninj4 commented 8 years ago

Hey, thanks for the offer. I've got most of them passing in this branch: https://github.com/holidayextras/jsonapi-client/pull/38 (I'll go ahead and merge it, I got distracted the other day). I've got a strange issue whereby the whole test suite passes locally but Safari fails when run against SauceLabs. I'm not sure about it, I may have to drop their support team a message and see if they have any ideas.

Running Karma locally should be a case of npm install && npm run karmalocal - give it a whirl in a sec once I've merged the test suite fixes. The SauceLabs keys are hidden in Travis environment variables, but you're welcome to push up changes in a pull request and have Travis run those for you. Let me know if you have any troubles.

gabesmed commented 8 years ago

thanks for the quick response! looking promising

gabesmed commented 8 years ago

hm, still failing for the test PR? https://travis-ci.org/holidayextras/jsonapi-client/builds/144013626

theninj4 commented 8 years ago

Looks like SauceLabs is having issues:

11 07 2016 22:00:16.285:INFO [launcher]: Launching browsers win10chrome, androidChrome, win10firefox, osxSafari, iosSafari, iosSafari92, win10ie11, win7ie9 with concurrency 5
11 07 2016 22:00:16.315:INFO [launcher]: Starting browser chrome (Windows 10) on SauceLabs
11 07 2016 22:00:16.346:INFO [launcher]: Starting browser android (Linux) on SauceLabs
11 07 2016 22:00:16.347:INFO [launcher]: Starting browser firefox (Windows 10) on SauceLabs
11 07 2016 22:00:16.347:INFO [launcher]: Starting browser safari (OS X 10.11) on SauceLabs
11 07 2016 22:00:16.347:INFO [launcher]: Starting browser iphone (OS X 10.10) on SauceLabs
11 07 2016 22:00:31.904:ERROR [launcher.sauce]: Can not start chrome (Windows 10)
  Failed to start Sauce Connect:
  Could not start Sauce Connect. Exit code 1 signal: null
11 07 2016 22:00:31.906:ERROR [launcher.sauce]: Can not start android (Linux)
  Failed to start Sauce Connect:
  Could not start Sauce Connect. Exit code 1 signal: null
11 07 2016 22:00:31.907:ERROR [launcher.sauce]: Can not start firefox (Windows 10)
  Failed to start Sauce Connect:
  Could not start Sauce Connect. Exit code 1 signal: null
11 07 2016 22:00:31.908:ERROR [launcher.sauce]: Can not start safari (OS X 10.11)
  Failed to start Sauce Connect:
  Could not start Sauce Connect. Exit code 1 signal: null
11 07 2016 22:00:31.908:ERROR [launcher.sauce]: Can not start iphone (OS X 10.10)
  Failed to start Sauce Connect:
  Could not start Sauce Connect. Exit code 1 signal: null

It's nearly midnight where I am, I'll restart your build in the morning.

If everything works, the test suite will fail but just above the mocha fail spec's you'll see:

Chrome 51.0.2704 (Windows 10 0.0.0): Executed 27 of 29 (skipped 2) SUCCESS (10.163 secs / 9.151 secs)
Firefox 47.0.0 (Windows 10 0.0.0): Executed 27 of 29 (skipped 2) SUCCESS (13.075 secs / 11.617 secs)
Safari 9.0.0 (Mac OS X 10.11.0): Executed 20 of 29 (19 FAILED) (skipped 2) (19.438 secs / NaN secs)
IE 11.0.0 (Windows 10 0.0.0): Executed 27 of 29 (skipped 2) SUCCESS (8.537 secs / 7.967 secs)
Mobile Safari 7.0.0 (iOS 7.1.0): Executed 27 of 29 (skipped 2) SUCCESS (9.403 secs / 8.651 secs)
IE 9.0.0 (Windows 7 0.0.0): Executed 27 of 29 (skipped 2) SUCCESS (6.904 secs / 6.084 secs)
Chrome Mobile 39.0.0 (Android 5.1.0): Executed 27 of 29 (skipped 2) SUCCESS (12.345 secs / 11.203 secs)
Mobile Safari 9.0.0 (iOS 9.2.0): Executed 20 of 29 (19 FAILED) (skipped 2) (10.327 secs / NaN secs)

^^ Safari in SauceLabs refuses to play ball. Everything else is working.

theninj4 commented 8 years ago

Hmm I've got no idea why SauceLabs isn't running the tests via Travis. The builds clearly aren't running according to the open source saucelabs page https://saucelabs.com/u/theninj4

SauceLabs is working fine locally with the private keys in place, just not on Travis. It was working fine a few days ago, I'm inclined to give them a few days and see if it sorts itself out.

Don't be too put off by the red light - the tests run in three parts:

  1. Mocha tests (the test suite run in NodeJS)
  2. Karma + Firefox tests (the same test suite in Firefox)
  3. Karma + SauceLabs (the same test suite in all the browsers)

If the first two steps pass, then you can be confident the changes are solid. The builds are currently failing on step 3, which is tricky because I'm using the open-source free offerings of Travis and SauceLabs so shouldn't complain too much when it gets a little shaky from time to time...