jejacks0n / teaspoon

Teaspoon: Javascript test runner for Rails. Use Selenium, BrowserStack, or PhantomJS.
1.43k stars 244 forks source link

BrowserStack Integration #458

Closed vedharish closed 8 years ago

vedharish commented 8 years ago

This PR adds a 'browserstack' driver which can be used to run tests on browserstack.

Additional support in spec/teaspoon_env.rb :

config.driver = :browserstack
config.driver_options = {
  capabilities: [{
    os: "OS X",
    os_version: "El Capitan",
    browser: "chrome",
    browser_version: ""
  }, {
    os: "Windows",
    os_version: "10",
    browser: "firefox",
    browser_version: "44"
  }],
  username: ENV['BROWSERSTACK_USERNAME'],
  access_key: ENV['BROWSERSTACK_ACCESS_KEY'],
  project: 'BrowserStack Project Name'
  build: 'BrowserStack Build Name',
}

The capabilities object is send as is to BrowserStack as desired_capabilities.

To access locally served pages on BrowserStack, we need to run a platform specific binary in the background. So, will have to run the binary from the command-line before running tests.

jejacks0n commented 8 years ago

Thanks! This is great, and all your work here looks really good. I'll do an in depth dive later today or this weekend, but from a glance it looks like you've done some really hard and good work here. You noticed hound is doing its job trying to keep up some code consistency rules, and it looks like you're cleaning those up.

For this to be merged, I'd like to see a unit test of the new BrowserStack implementation so we can make sure not to break it in the future (an example: https://github.com/modeset/teaspoon/blob/master/spec/teaspoon/driver/selenium_spec.rb), and it looks like there's a broken spec around the CLI help -- the new browserstack driver correctly appears in the options displayed when running teaspoon --help, so the expectation in that spec should change to ensure it stays there. =)

As a follow up to this PR, I'd appreciate it if you updated the README, and added a wiki article with similar information to your PR description -- any third party things like browserstack can be complex to setup, so any information you have would be really valuable there.

If there's anything I can do to help, let me know.

jejacks0n commented 8 years ago

As a follow up, when this is merged, I will update things like the repo description to include BrowserStack as a viable driver setup.

vedharish commented 8 years ago

Thank you for being patient.

I have added the unit tests for the added driver. You might want to go through those.

Regarding the BrowserStackLocal binary - There are some ways to go about this. Issue - The tests require this platform specific binary running in the background.

Pending: The wiki page for using browserstack driver. Will proceed after the above discussion is clear.

PS: Is there a way to run the houndci tests locally? (Before doing an actual push). I couldn't find one.

jejacks0n commented 8 years ago

Ok, I'm out for a few days (surgery), but will review after that. I really appreciate your time.

To answer your questions: Let's use option 3. That's the same as we do for selenium, phantom etc. There's some tooling in util that allows us to find that binary if it's in the path, so probably want to use that same process here.

No, there's no a nice way to check code styles without hound -- I have the rubocop config locally I think, but never pushed that up. Don't worry about them, it's not a bad thing, we just want to keep relatively consistent code style so you can just clean them up as you want. Those emails don't bother me; just it doing its job. :)


On Feb 29, 2016, at 3:31 AM, Harish Ved notifications@github.com wrote:

Thank you for being patient.

I have added the unit tests for the added driver. You might want to go through those.

Regarding the BrowserStackLocal binary - There are some ways to go about this. Issue - The tests require this platform specific binary running in the background.

There will be a published gem for this purpose. The source code is here. It does start the binary, stops it and adds the logs of the binary to a file. Will be efficient to use. But it might take some time for it to be published on rubygems.org and we will have to add it as a dependency. A thin version of the gem file can be implemented here which downloads the binary (according to the platform its running on) and starts it before and stops it after tests. We can just add a word of warning that this requires a specific binary running. But running the tests for the users won't be automatic and will require setup from user's part. Pending: The wiki page for using browserstack driver. Will proceed after the above discussion is clear.

PS: Is there a way to run the houndci tests locally? (Before doing an actual push). I couldn't find one.

— Reply to this email directly or view it on GitHub.

jejacks0n commented 8 years ago

My last response was truncated, please expand it to see the hound comments.


On Feb 29, 2016, at 3:31 AM, Harish Ved notifications@github.com wrote:

Thank you for being patient.

I have added the unit tests for the added driver. You might want to go through those.

Regarding the BrowserStackLocal binary - There are some ways to go about this. Issue - The tests require this platform specific binary running in the background.

There will be a published gem for this purpose. The source code is here. It does start the binary, stops it and adds the logs of the binary to a file. Will be efficient to use. But it might take some time for it to be published on rubygems.org and we will have to add it as a dependency. A thin version of the gem file can be implemented here which downloads the binary (according to the platform its running on) and starts it before and stops it after tests. We can just add a word of warning that this requires a specific binary running. But running the tests for the users won't be automatic and will require setup from user's part. Pending: The wiki page for using browserstack driver. Will proceed after the above discussion is clear.

PS: Is there a way to run the houndci tests locally? (Before doing an actual push). I couldn't find one.

— Reply to this email directly or view it on GitHub.

vedharish commented 8 years ago

Will add this page as a follow-up to the PR

jejacks0n commented 8 years ago

Looks good! Thanks for the initial work, and being flexible and accommodating with the follow up requests.

I'll merge, give it some time on master, and then release the new gem version.

Follow up question: Do you work for BrowserStack, or was this something you wanted personally and were willing to share?

vedharish commented 8 years ago

I work for BrowserStack :) Thanks! :+1:

jejacks0n commented 8 years ago

Did you ever get that wiki page created? =)

vedharish commented 8 years ago

Ohh.. Just realised. I added it as https://github.com/modeset/teaspoon/wiki/Using-BrowserStack-Driver Just renamed it to https://github.com/modeset/teaspoon/wiki/Using-BrowserStack-WebDriver and added to the sidebar. :+1: