upgrade KGS

[gotcha]

All plone.app.robotframework tests pass. Same for collective.ckeditor tests...

[jensens]

@plone/testing-team which version of firefox is running at our jenkins setup?

[gforcada]

@jensens it is versin 46: https://github.com/plone/plone.jenkins_node/blob/master/defaults/main.yml

[gotcha]

Is there a way to prove that it would work on Jenkins with Firefox 57 ?

[gforcada]

@gotcha at least not right now with small effort, specially with firefox and selenium that starting from some recent releases the driver got split out of the firefox binary and the new one is not so stable...

[gotcha]

@gforcada Did you try latest versions ? It has stabilized a lot in my opinion.

[gforcada]

@gotcha no I did not, any pointers on how to install and configure it would of course speed it up 😃

[gforcada]

@gotcha I just tried locally and after installing the geckodriver I can run robot tests on my firefox 58 🎉

Now the question is: on jenkins nodes we can only have one version of Firefox installed, but upgrading it means that we need to install the geckodriver and bump selenium versions.

@plone/framework-team @plone/release-team would it be ok to do a major bump of versions on selenium and related packages on 4.3, 5.0 and 5.1 ? I would be 👍 on it as they are just testing dependencies, but that's only my opinion...

[jensens]

Imo, Yes, we should upgrade all supported Plone versions. Robottests must run against real environments, and after automatically upgrades of Firefox on Desktops are the gold standard now, almost nobody uses an outdated Firefox.

[jensens]

Also this would make local testing much more approachable.

[mauritsvanrees]

Fine with me to make PRs to bump versions in all coredev branches.

I have recently installed chromedriver, which means I can run robot tests again, so I started wondering if we should install that on Jenkins. But if geckodriver does the job, then that is fine with me.

I found these links which seem to explain the situation with firefox/geckodriver/selenium well:

• https://www.quora.com/Why-should-I-use-the-Gecko-Driver-in-Selenium-3-0
• https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver

[gforcada]

@jensens @mauritsvanrees thanks for the feedback!

@gotcha would you mind providing the pull requests on buildout.coredev on 4.3, 5.0, 5.1 and 5.2 to bump the same version pins you are changing here? Or should we do that ourselves?

@mauritsvanrees regarding Chrome and Chrome driver, we could of course add that as well, and then have tests first run in firefox and then with Chrome, I'm mostly concerned on the time it will take to run all of it, should we do that on the same job or rather have two different jobs (one for Chrome and one for Firefox)?

[mauritsvanrees]

@gforcada I think we should use one driver. Independently of this issue I was thinking of chromedriver because that is more recent than what we currently have. But if geckodriver (which I didn't know yet) works good, then we can use that instead.

We could run robot tests with a second browser. Maybe an extra job that only runs once a week. But if Plone works fine on Firefox, then it probably works fine on Chrome as well, and the other way around. It is much more likely that something works there, but goes wrong on IE or Edge.

[tisto]

+1 to all comments. We have to make sure to keep our drivers and browsers up to date.

[datakurre]

@gforcada @gotcha I'm sorry I'm late on this. I assume it is ok to merge this, because coredev has its own pins anyway?

I had the same feelings about geckodriver + Firefox, but since Firefox 57 was released, I have no longer had issues it that personally. Also, since the latest selenium[2]library release, also chromeheadless and firefoxheadless are supported directly (obviously either chromedriver or geckodriver must still exist).

[hvelarde]

I was able to run collective.cover tests sucessfully with this KGS; the only issue now is to define a canonical way to install geckodriver as Travis doesn't provide one.

[hvelarde]

@plone/framework-team tests are passing in all Plone versions, but 4.3; IMO this can be merged now.

@datakurre can you please check why tests are falling on Travis?

[ale-rt]

@gotcha I think this can be closed because on buildout coredev we already have newer versions: https://github.com/plone/buildout.coredev/blob/c2c5fcf978d51581e61725e0373aeb9b376cfdfc/versions.cfg#L110-L119