search

open-lms-open-source / moodle-plugin-ci

Assist with running a Moodle plugin in Travis CI
https://blackboard-open-source.github.io/moodle-plugin-ci/
GNU General Public License v3.0
42 stars 37 forks source link

Support cross-DB automated tests with moodle-docker #57

Open danpoltawski opened 7 years ago

danpoltawski commented 7 years ago

It would be great if plugin authors could run their unit/behat tests against all supported DB drivers with moodle-plugin-ci.

With https://github.com/moodlehq/moodle-docker it is quite straight forward to do this in the travis environment. I have created an example travis file to show how you can achieve this with a simple mod_workshop example. (Results)

Unfortunately I am not able to work on fitting this into the moodle-plugin-ci infrastructure, but I think it would be a really great addition to improve cross-db compatability across plugins.

https://github.com/danpoltawski/moodle/blob/travis-plugin-test/.travis.yml https://travis-ci.org/danpoltawski/moodle/builds/277265066

polothy commented 7 years ago

This is very cool Dan, really shows off the power of Docker.

Little concerned about performance though, looks like it takes 400+ seconds to initialize just Behat or PHPUnit, while currently it takes around 80 seconds for me to init both while also installing several other things (Selenium, NPM modules, etc). I also like the portability of MPCI as we can run it in our Vagrant box as well as in Travis.

Perhaps your suggestion is just for mssql and oracle, which makes sense, maybe that can be done. Don't really like the hybrid approach though since it creates multiple (very different) paths to support. Have to think about it some more...

Thanks again for the suggestion, it is very cool.

danpoltawski commented 7 years ago

I agree performance is going to be a tricky thing to consider that startup cost is high - it's never going to be as fast as doing it 'natively' because you will need to download the images and oracle/mssql are slow to start up. So in that sense I think a hybrid approach has merit (it also reduces the complexity of changing to doing everything inside containers). I was envisaging something like a cross-db phpunit/behat task which runs as a seperate job in the matrix (can be in parallel to the other jobs).

There are some optimisations which can be done on the moodle-docker side to reduce the amount of services/containers to the bare minimum for tests, but in the grand scheme of things its still going to be slower whatever you do.