Not able to run build with different version of firefox

rturak commented 8 years ago

Is your request:

[ ] A feature request: The feature you want currently does not exist. If implemented, will provide additional value to you.
[ ] A technical issue: You are unable to use Shippable's service per your expectation & need help to resolve the issue.
[ ] A question: A non-technical question such as billing.
Description of your issue:

NOTE: If you are experiencing a build failure, please include:

Link to the build failure
Scripts called from the yml (optional)

a-murphy commented 8 years ago

The addons section doesn't support multiple versions right now. I'll mark this as a feature request.

For now, if you need multiple versions in your matrix, you will have to put the versions that you want in the env section and start them yourself in your build script. The commands that we run to start Firefox can be found in the install_firefox_addon section of your build logs.

rturak commented 8 years ago

Thanks Murphy for quick reply.

My goal is to run test different versions browser in one build.

Here is the yml file:

`language: java

jdk:

oraclejdk8

addons: selenium: "2.53"

services:

selenium

env:

firefox="45.0"
firefox="44.0"

before_install:

"curl -o /tmp/chromedriver.zip -L http://chromedriver.storage.googleapis.com/2.21/chromedriver_linux64.zip"
"unzip /tmp/chromedriver.zip -d /home/shippable/bin"
"chmod +x /home/shippable/bin/chromedriver"
"echo '#!/bin/bash' | sudo tee /usr/local/bin/google-chrome"
"echo 'exec /usr/local/bin/chrome --no-sandbox --disable-gpu \"$@\"' | sudo tee -a /usr/local/bin/google-chrome"
"sudo chmod +x /usr/local/bin/google-chrome"

script:

sudo ln -sf /tmp/firefox/firefox /usr/bin/firefox
export DISPLAY=:99.0
/etc/init.d/xvfb start
mvn clean cobertura:cobertura
mvn test`

Error:

Running TestSuite
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 0.533 sec <<< FAILURE! - in TestSuite
setUp(net.lacework.SignIn)  Time elapsed: 0.419 sec  <<< FAILURE!
org.openqa.selenium.WebDriverException: Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '31acb1b8c2aa', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: driver.version: FirefoxDriver
    at org.openqa.selenium.firefox.internal.Executable.<init>(Executable.java:75)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:60)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:56)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:120)
    at net.lacework.SignIn.setUp(SignIn.java:38)

This error is because ff is not installed as it is not in addon section. But if i add ff version in addon section then every build will have the same version of ff installed. That is not what i want.

a-murphy commented 8 years ago

I don't think Selenium can run without a browser. You'll have to take it out of services, and move the commands from selenium_start into your script (after installing Firefox). I also don't see where in your script you are installing the version of Firefox that you plan to use.

rturak commented 8 years ago

If i add firefox version to addon section, then for very build i will get the same version of ff installed. But i want to test different versions of ff.

rturak commented 8 years ago

If i provide the version of firefox in the script , its same issue. Both the builds will run with same firefox version. Not sure by what you mean "take it out of services". Could you please help with yml if i am missing your instructions.

a-murphy commented 8 years ago

I was thinking something more like this:

language: java

jdk:
  - oraclejdk8

addons:
  selenium: "2.53"

env:
  - firefox="45.0"
  - firefox="44.0"

before_install:
 # Firefox
  - "sudo wget \"https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/$firefox/linux-x86_64/en-US/firefox-$firefox.tar.bz2\""
  - sudo tar -xjvf firefox-$firefox.tar.bz2
  - sudo ln -sf /tmp/firefox/firefox /usr/bin/firefox
  - ff_version=`firefox --version`
  - echo "Using firefox version $ff_version"

  # Selenium
  - export SHIPPABLE_SELENIUM_PORT=4444
  - export DISPLAY=":99.0"
  - start_generic_service "selenium" "$SHIPPABLE_SELENIUM_BINARY" "$SHIPPABLE_SELENIUM_CMD" "$SHIPPABLE_SELENIUM_PORT" "$SHIPPABLE_SELENIUM_LOG";

  - "curl -o /tmp/chromedriver.zip -L http://chromedriver.storage.googleapis.com/2.21/chromedriver_linux64.zip"
  - "unzip /tmp/chromedriver.zip -d /home/shippable/bin"
  - "chmod +x /home/shippable/bin/chromedriver"
  - "echo '#!/bin/bash' | sudo tee /usr/local/bin/google-chrome"
  - "echo 'exec /usr/local/bin/chrome --no-sandbox --disable-gpu \"$@\"' | sudo tee -a /usr/local/bin/google-chrome"
  - "sudo chmod +x /usr/local/bin/google-chrome"

script:
  - export DISPLAY=:99.0
  - /etc/init.d/xvfb start
  - mvn clean cobertura:cobertura
  - mvn test

  # Stop Selenium
  - sudo su -c "killall -15 java";

rturak commented 8 years ago

Firefox is installed but still getting error that not able to find firefox binary.I got this error:

Running TestSuite
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 0.584 sec <<< FAILURE! - in TestSuite
setUp(net.lacework.SignIn)  Time elapsed: 0.419 sec  <<< FAILURE!
org.openqa.selenium.WebDriverException: Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '60f86249bdf3', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: driver.version: FirefoxDriver
    at org.openqa.selenium.firefox.internal.Executable.<init>(Executable.java:75)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:60)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:56)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:120)
    at net.lacework.SignIn.setUp(SignIn.java:38)

a-murphy commented 8 years ago

Could you try xvfb-run --server-args="-ac" mvn test instead of /etc/init.d/xvfb start? xvfb is known to be more likely to work in our images than /etc/init.d/xvfb.

rturak commented 8 years ago

I have added PATH for firefox and xvfb-run --server-args="-ac" mvn test Here is the yml:

language: java

jdk:
  - oraclejdk8

addons:
  selenium: "2.53"

env:
  - firefox="45.0"
  - firefox="44.0"
  - firefox="43.0"
  - firefox="42.0"
  - firefox="41.0"

before_install:
 # Firefox
  - "sudo wget \"https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/$firefox/linux-x86_64/en-US/firefox-$firefox.tar.bz2\""
  - sudo tar -xjvf firefox-$firefox.tar.bz2
  - sudo ln -sf /tmp/firefox/firefox /usr/bin/firefox
  - export PATH=$PATH:/usr/bin/firefox/firefox
  - echo $PATH
  # - ff_version='firefox --version 2>&1';
  # - echo "Using firefox version $ff_version"

  # Selenium
  - export SHIPPABLE_SELENIUM_PORT=4444
  - export DISPLAY=":99.0"
  - start_generic_service "selenium" "$SHIPPABLE_SELENIUM_BINARY" "$SHIPPABLE_SELENIUM_CMD" "$SHIPPABLE_SELENIUM_PORT" "$SHIPPABLE_SELENIUM_LOG";

  - "curl -o /tmp/chromedriver.zip -L http://chromedriver.storage.googleapis.com/2.21/chromedriver_linux64.zip"
  - "unzip /tmp/chromedriver.zip -d /home/shippable/bin"
  - "chmod +x /home/shippable/bin/chromedriver"
  - "echo '#!/bin/bash' | sudo tee /usr/local/bin/google-chrome"
  - "echo 'exec /usr/local/bin/chrome --no-sandbox --disable-gpu \"$@\"' | sudo tee -a /usr/local/bin/google-chrome"
  - "sudo chmod +x /usr/local/bin/google-chrome"

after_script:
  - export DISPLAY=:99.0
  # - /etc/init.d/xvfb start
  - mvn clean cobertura:cobertura
  - xvfb-run --server-args="-ac" mvn test

  # Stop Selenium
  - sudo su -c "killall -15 java";

Here is the error:

T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 0.515 sec <<< FAILURE! - in TestSuite
setUp(net.lacework.SignIn)  Time elapsed: 0.383 sec  <<< FAILURE!
org.openqa.selenium.WebDriverException: Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '35ee05373684', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: driver.version: FirefoxDriver
    at org.openqa.selenium.firefox.internal.Executable.<init>(Executable.java:75)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:60)
    at org.openqa.selenium.firefox.FirefoxBinary.<init>(FirefoxBinary.java:56)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:120)
    at net.lacework.SignIn.setUp(SignIn.java:38)

Results :

Failed tests: 
  SignIn.setUp:38 » WebDriver Cannot find firefox binary in PATH. Make sure fire...

Tests run: 3, Failures: 1, Errors: 0, Skipped: 2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.872 s
[INFO] Finished at: 2016-04-28T20:53:07+00:00
[INFO] Final Memory: 12M/106M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project RainbowRegressionTestSuite: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/shippable/workspace/src/github.com/RainbowRegressionTestSuite/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

a-murphy commented 8 years ago

Have you tried which firefox? The path shouldn't be a problem, unless your tests are specifically looking for it somewhere else.

rturak commented 8 years ago

No my test is not looking for any specific path. I was able to run tests before the changes.

a-murphy commented 8 years ago

Sorry, it looks like I copied the wrong version of my shippable.yml earlier. The line to get the version of Firefox should just be ff_version='firefox --version', but it doesn't matter if it's commented out. It does matter that the pushd /tmp and popd around the Firefox install went missing. It causes strange errors (I think it ends up in the wrong directory). Here's what that should have been:

  # Firefox
  - pushd /tmp
  - "sudo wget \"https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/$firefox/linux-x86_64/en-US/firefox-$firefox.tar.bz2\""
  - sudo tar -xjvf firefox-$firefox.tar.bz2
  - sudo ln -sf /tmp/firefox/firefox /usr/bin/firefox
  - ff_version=`firefox --version`
  - echo "Using firefox version $ff_version"
  - popd

rturak commented 8 years ago

Thanks for your reply. I noticed that path was wrong so i tried below which worked for me:

before_install:
 #xvfb
  - export DISPLAY=:99.0
  - /etc/init.d/xvfb start
 # Firefox
  - "sudo wget \"https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/$firefox/linux-x86_64/en-US/firefox-$firefox.tar.bz2\""
  - sudo tar -xjvf firefox-$firefox.tar.bz2
  - sudo ln -sf /usr/lib/firefox/firefox /usr/bin/firefox

But now i am getting no such element error, i know its nothing to do with this. But this was earlier working for me. I am also able to run it through my local set up.

Here is the error:


 T E S T S
-------------------------------------------------------
Running TestSuite
Opened Application
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 53.085 sec <<< FAILURE! - in TestSuite
testLogin(net.lacework.SignIn)  Time elapsed: 43.806 sec  <<< FAILURE!
org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"class name","selector":"input-text"}
Command duration or timeout: 30.02 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '13f4d2e55c2d', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{applicationCacheEnabled=true, rotatable=false, handlesAlerts=true, databaseEnabled=true, version=33.0, platform=LINUX, nativeEvents=false, acceptSslCerts=true, webStorageEnabled=true, locationContextEnabled=true, browserName=firefox, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
Session ID: dc7c990d-0c37-45dd-acd8-4d3369333d31
*** Element info: {Using=class name, value=input-text}
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
    at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:678)
    at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:363)
    at org.openqa.selenium.remote.RemoteWebDriver.findElementByClassName(RemoteWebDriver.java:477)
    at org.openqa.selenium.By$ByClassName.findElement(By.java:391)
    at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:355)
    at net.lacework.SignIn.testLogin(SignIn.java:60)
Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"class name","selector":"input-text"}
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '13f4d2e55c2d', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: driver.version: unknown
    at <anonymous class>.FirefoxDriver.prototype.findElementInternal_(file:///tmp/anonymous7107899016824098469webdriver-profile/extensions/fxdriver@googlecode.com/components/driver-component.js:10770)
    at <anonymous class>.fxdriver.Timer.prototype.setTimeout/<.notify(file:///tmp/anonymous7107899016824098469webdriver-profile/extensions/fxdriver@googlecode.com/components/driver-component.js:625)

Results :

Failed tests: 
  SignIn.testLogin:60 » NoSuchElement Unable to locate element: {"method":"class...

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:02 min
[INFO] Finished at: 2016-04-29T00:12:30+00:00
[INFO] Final Memory: 13M/106M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project RainbowRegressionTestSuite: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/shippable/workspace/src/github.com/RainbowRegressionTestSuite/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Nothing changed in the test. I have also tried to rerun test after increasing sleep period

rturak commented 8 years ago

I am able to run different versions of ff now. But started getting error for chrome:

Here i am installing chrome(yml):

before_install:
 #xvfb
  - export DISPLAY=:99.0
  - /etc/init.d/xvfb start
 # Firefox
  - pushd /tmp
  - "sudo wget \"https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/$firefox/linux-x86_64/en-US/firefox-$firefox.tar.bz2\""
  - sudo tar -xjvf firefox-$firefox.tar.bz2
  - sudo ln -sf /tmp/firefox/firefox /usr/bin/firefox
  - ff_version=`firefox --version`
  - echo "Using firefox version $ff_version"
  - popd
# Selenium
  - export SHIPPABLE_SELENIUM_PORT=4444
  - export DISPLAY=":99.0"
  - start_generic_service "selenium" "$SHIPPABLE_SELENIUM_BINARY" "$SHIPPABLE_SELENIUM_CMD" "$SHIPPABLE_SELENIUM_PORT" "$SHIPPABLE_SELENIUM_LOG";

# Chrome
  - "curl -o /tmp/chromedriver.zip -L http://chromedriver.storage.googleapis.com/2.18/chromedriver_linux64.zip"
  - "unzip /tmp/chromedriver.zip -d /home/shippable/bin"
  - "chmod +x /home/shippable/bin/chromedriver"
  - "echo '#!/bin/bash' | sudo tee /usr/bin/google-chrome"
  - "echo 'exec /usr/bin/chrome --no-sandbox --disable-gpu \"$@\"' | sudo tee -a /usr/local/bin/google-chrome"
  - "sudo chmod +x /usr/local/bin/google-chrome"

And provided /home/shippable/bin in the test.

Here is the error:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
Starting ChromeDriver 2.21.371461 (633e689b520b25f3e264a2ede6b74ccc23cb636a) on port 25193
Only local connections are allowed.
Opened Application
Tests run: 4, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 65.219 sec <<< FAILURE! - in TestSuite
setUp(net.lacework.SignIn)  Time elapsed: 50.029 sec  <<< FAILURE!
org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally
  (Driver info: chromedriver=2.21.371461 (633e689b520b25f3e264a2ede6b74ccc23cb636a),platform=Linux 3.19.0-51-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 61.62 seconds
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: '0d67410d1ffc', ip: '172.17.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-51-generic', java.version: '1.8.0_25'
Driver info: org.openqa.selenium.chrome.ChromeDriver
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
    at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:678)
    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:249)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:131)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:144)
    at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:170)
    at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:138)
    at net.lacework.SignIn.setUp(SignIn.java:50)

Results :

Failed tests: 
  SignIn.setUp:50 » WebDriver unknown error: Chrome failed to start: exited abno...

Tests run: 4, Failures: 1, Errors: 0, Skipped: 2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:08 min
[INFO] Finished at: 2016-04-29T17:36:31+00:00
[INFO] Final Memory: 12M/106M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project RainbowRegressionTestSuite: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/shippable/workspace/src/github.com/RainbowRegressionTestSuite/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Please let me know if something is missing. I have referred all the chrome related issues and tried to do that, nothing helped.

a-murphy commented 8 years ago

I haven't been able to get Selenium to work with Chrome on that image yet. You might have more luck with one of the newer images with Chrome installed, but I can't be sure what you are relying on in the image you are currently using.

avinci commented 7 years ago

closing this as its inactive for a while. Please re-open if its not resolved yet

Shippable / support

Not able to run build with different version of firefox #2576

Is your request:

Description of your issue: