Selenium server is not running

[mackensen]

This is a new project to which I've just added Behat tests, but selenium doesn't start: https://travis-ci.org/mackensen/moodle-tool_coursedates/builds/255679371. Here's the raw output:

$ moodle-plugin-ci behat
 RUN  Behat features for tool_coursedates
Running single behat site:
Moodle 3.2.4+ (Build: 20170714), bc60c1a2eb8bc21d78494faf1b9bda0e7042ad7f
Php: 5.6.24, pgsql: 9.3.17, OS: Linux 4.8.12-040812-generic x86_64
Server OS "Linux", Browser: "firefox"
Started at 20-07-2017, 22:28
Selenium server is not running, you need to start it to run tests that involve Javascript. More info in http://docs.moodle.org/dev/Acceptance_testing#Running_tests

[polothy]

Are you able to run them locally? The Selenium server is not running, you need to start it to run tests that involve Javascript. is one of the worst errors because several things can cause that error, selenium problem, PHP server problem, any sort of debugging/error on the homepage, etc.

[mackensen]

I'm able to run the test locally, but selenium still fails to start on travis. No debugging/notices on the homepage.

[mackensen]

As a test I moved my local environment up to Selenium 2.53 and Java 8, matching travis. Same result--tests run just fine locally.

[polothy]

Had a co-worker have the same problem, but with v1. One thing that might help is to give more time to let Selenium startup. I increased it from 5 to 10 seconds. You can try it by changing:

  - composer create-project -n --no-dev --prefer-dist moodlerooms/moodle-plugin-ci ci ^2

To

  - composer create-project -n --no-dev --prefer-dist moodlerooms/moodle-plugin-ci ci dev-develop

[kiklop74]

The issue here is due to a fact that Travis is switching to Ubuntu Trusty (14.04) hence the default ci plugin assumptions are no longer valid. My investigation so far shows that there are three ways of resolving this.

Using FirefoxDriver

Moodle CI plugin uses selenium 2.x (v1 uses 2.4x and v2 2.53.x) hence in order to work firefox can not be of version larger than 45. New trusty image in Travis CI ships with custom firefox build located in /us/local/bin/firefox that is firefox 50. So that one does not work. Furthermore due to different path selenium server has to know explicitly where the binary is. Unfortunately neither v1 or v2 of the plugin support that setting. So we have to start selenium manually. Here is the example:

    language: php

    sudo: true

    cache:
      directories:
        - $HOME/.composer/cache

    services:
      - mysql

    addons:
      apt:
        packages:
         - firefox=28.0+build2-0ubuntu2

    env:
      global:
        - MOODLE_START_BEHAT_SERVERS=NO
        - DB=mysqli
        - MOODLE_CI_VER=1
        - MOODLE_DIR=/home/travis/build/moodle
        - MOODLE_SELENIUM_JAR=/home/travis/build/moodle/selenium.jar

    matrix:
     include:
      - php: 7.0
        env: MOODLE_BRANCH=MOODLE_32_STABLE MOODLE_CI_VER=2

    before_install:
      - phpenv config-rm xdebug.ini
      - cd ../..
      - composer selfupdate
      - composer create-project -n --no-dev --prefer-dist moodlerooms/moodle-plugin-ci ci ^${MOODLE_CI_VER}
      - export PATH="$(cd ci/bin; pwd):$(cd ci/vendor/bin; pwd):$PATH"
      - sleep 10

    install:
      - moodle-plugin-ci install
      - xvfb-run -a --server-args="-screen 0 1024x768x24" java -Dwebdriver.firefox.bin=/usr/bin/firefox -jar ${MOODLE_SELENIUM_JAR} -log /tmp/selenium.log &
      - php -t ${MOODLE_DIR} -S localhost:8000 > /tmp/php-access.log 2> /tmp/php-error.log &
      - sleep 5s

    script:
      - moodle-plugin-ci phplint
      - moodle-plugin-ci codechecker
      - moodle-plugin-ci phpunit
      - moodle-plugin-ci behat --moodle=${MOODLE_DIR} --auto-rerun=0

As you can see we install explicitly old version of firefox 28 from Ubuntu repositories and than in the selenium CLI explicitly set path to the binary. We also need to explicitly specify path to Moodle for behat since it assumes the current dir is Moodle dir and may throw error.

Using Chromium Driver

Since Travis version of trusty image already comes with Chromium preinstalled we can also use that for testing. Script would look like this:

    language: php

    sudo: true

    cache:
      directories:
        - $HOME/.composer/cache

    services:
      - mysql

    addons:
      apt:
        packages:
         - chromium-chromedriver
    env:
      global:
        - MOODLE_START_BEHAT_SERVERS=NO
        - DB=mysqli
        - MOODLE_CI_VER=1
        - MOODLE_DIR=/home/travis/build/moodle
        - MOODLE_SELENIUM_JAR=/home/travis/build/moodle/selenium.jar

    matrix:
     include:
      - php: 7.0
        env: MOODLE_BRANCH=MOODLE_32_STABLE MOODLE_CI_VER=2

    before_install:
      - phpenv config-rm xdebug.ini
      - cd ../..
      - composer selfupdate
      - composer create-project -n --no-dev --prefer-dist moodlerooms/moodle-plugin-ci ci ^${MOODLE_CI_VER}
      - export PATH="$(cd ci/bin; pwd):$(cd ci/vendor/bin; pwd):$PATH"
      - sleep 10

    install:
      - moodle-plugin-ci install
      - xvfb-run -a --server-args="-screen 0 1024x768x24" java -Dwebdriver.chrome.driver=/usr/lib/chromium-browser/chromedriver -jar ${MOODLE_SELENIUM_JAR} -log /tmp/selenium.log &
      - php -t ${MOODLE_DIR} -S localhost:8000 > /tmp/php-access.log 2> /tmp/php-error.log &
      - sleep 5s

    script:
      - moodle-plugin-ci phplint
      - moodle-plugin-ci codechecker
      - moodle-plugin-ci phpunit
      - moodle-plugin-ci behat --moodle=${MOODLE_DIR} --auto-rerun=0

Using Mozilla Gecko driver

Mozilla gecko driver is the latest version of selenium support that handles firefox 54 or more recent. Here we have to download latest selenium server (3.4), mozilla geckodriver and install firefox 54.

    language: php

    sudo: true

    cache:
      directories:
        - $HOME/.composer/cache

    services:
      - mysql

    addons:
      apt:
        packages:
         - firefox

    env:
      global:
        - MOODLE_START_BEHAT_SERVERS=NO
        - DB=mysqli
        - MOODLE_CI_VER=1
        - MOODLE_DIR=/home/travis/build/moodle
        - MOODLE_SELENIUM_JAR=/home/travis/build/moodle/selenium34.jar

    matrix:
     include:
      - php: 7.0
        env: MOODLE_BRANCH=MOODLE_32_STABLE MOODLE_CI_VER=2

    before_install:
      - phpenv config-rm xdebug.ini
      - cd ../..
      - composer selfupdate
      - composer create-project -n --no-dev --prefer-dist moodlerooms/moodle-plugin-ci ci ^${MOODLE_CI_VER}
      - export PATH="$(cd ci/bin; pwd):$(cd ci/vendor/bin; pwd):$PATH"
      - sleep 10

    install:
      - moodle-plugin-ci install
      - curl -L -s -o ${MOODLE_DIR}/selenium34.jar http://selenium-release.storage.googleapis.com/3.4/selenium-server-standalone-3.4.0.jar
      - curl -L -s -o /tmp/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/v0.18.0/geckodriver-v0.18.0-linux64.tar.gz
      - sudo tar -C /usr/local/bin/ -xzf /tmp/geckodriver.tar.gz
      - xvfb-run -a --server-args="-screen 0 1024x768x24" java -Dwebdriver.gecko.driver=/usr/local/bin/geckodriver -Dwebdriver.firefox.bin=/usr/bin/firefox -Dwebdriver.firefox.marionette=1 -jar ${MOODLE_SELENIUM_JAR} -log /tmp/selenium.log &
      - php -t ${MOODLE_DIR} -S localhost:8000 > /tmp/php-access.log 2> /tmp/php-error.log &
      - sleep 5s

    script:
      - moodle-plugin-ci phplint
      - moodle-plugin-ci codechecker
      - moodle-plugin-ci phpunit
      - moodle-plugin-ci behat --moodle=${MOODLE_DIR} --auto-rerun=0

Conclusion

This solves the problem but it would probably be better if CI plugin itself would permit passing of custom binary paths for browsers and drivers etc.

[kiklop74]

Also we can enforce CI to use precise instead of trusty with:

dist: precise

sudo: true

[kiklop74]

I tested with precise and Moodle CI v2 fails to execute due to too old version of nodejs. Build log is here https://travis-ci.org/kiklop74/moodle-local_xray/builds/256590809

Fatal error: Node version too old. Require >=4.0.0, version installed: 0.10.36

[kiklop74]

Confirmed. With precise only Moodle CI v1 works.

[polothy]

This should be fixed in 2.1.0, BUT you must take action to get the fix, see https://moodlerooms.github.io/moodle-plugin-ci/CHANGELOG.html

The same fix should likely work with v1 as well.