Multiple environments

[tobiastom]

It would be awesome if I could run e.g. PHPUnit tests with PHP 5.3.x, 5.4.x and 5.5.x. Travis does support this feature and I really would like to see it on PHPCI.

I have no clue how to implement it best. Travis does it with separate servers, I guess.

[meadsteve]

I believe Travis does it by setting up small virtual machines for each environment you want to test. It'd be a fairly chunky piece of work to do but would give us a lot more control over (and isolation of) the environment actually running the commands.

Another advantage would be it would remove a lot of the problems we've had when running on windows vs linux as everything would be running on a vm.

[meadsteve]

I'd be tempted to suggest vagrant (http://www.vagrantup.com/) for this but then we've added a ruby dependency which is possibly counter to the aims of this project.

[tobiastom]

There are more solutions like vagrant, and I think all of them would be a good solution. The main point with this would be that I do not want to install PHPCI on every different environment and check multiple websites.

I'd like to have one central place to see which build worked on which environment.

[gabriel403]

I don't think we need to worry about introducing more dependencies, we need to provide the best experience for the end user, the people running the tests, as we can, if that means we need to install vagrant too, I don't think it's a problem, the base os probably comes with ruby in any case.

[dancryer]

This was something I've wanted to do from the beginning (it's even in the README) - but as @meadsteve says, it's a big piece of work, so we really need to get it right.

My suggestion would be that it would always be optional, so by default the tests run on the PHPCI host. My preference for making it actually work would be either Vagrant or Docker (docker.io for reference.)

Vagrant is the tool I am more familiar with, and I actually already have a Vagrantfile for PHPCI, so the change could be relatively simple in that regard, as long as the MySQL server is referenced by a non-localhost address that the VM can access, you could just run the PHPCI console application as normal. We would just need to add some sort of support for sub-builds or similar.

With that being said, from what I understand, Docker is significantly more lightweight - so it might be worth investigating that instead. Docker can even run inside a Vagrant VM.

[dancryer]

P.s. I don't mind adding dependencies like this as long as they are optional. Not everyone needs to do multi-version testing, so PHPCI should continue to function as it does now for those users - that's my only requirement.

[tobiastom]

Regarding dependencies: I agree with you, but then I'm wondering why e.g. phpmd is a hard dependency, when I might not use it? IMHO "we" should make them optional as well.

Regarding Vagrant, Docker and whatever: I'm in favor of some VM stuff, BUT: Please do not depend on one service. I think we should search for a solution that can be used with own stuff as well. So for example we might define some communication between a test environment and PHPCI. This could be used by a VM, but this could also be used by a test server I set up once and reuse it.

This would also add a nice side-effect: we could run the build server on a production box so have that checked as well. Let's not make it about of that makes sense in a real world, but it is a great example about the flexibility it could provide us.

Also I'm strongly against exposing MySQL to the outside world. IMHO we should define some REST API stuff for communication between PHPCI and… "build environment".

[dancryer]

The dependencies thing has come up before, and I agree, PHPCI is already a different project to what it was when I started it a few months ago. At the time it made sense for it to bundle PHPMD, PHPUnit and so on because there were only about 4 plugins. Now there are more like 20 it's unreasonable for it to install everything it could need.

As for your point about using one service, I totally agree. My initial thinking was that you'd define, either in the web interface or at install time, a set of build servers - and that they could be any kind of server - but that it would also provide a method of defining, say, a Vagrant .box file that it should launch.

As for exposing MySQL, that wasn't my intent - but I do quite like the idea of a builder API. My initial thinking was that you'd simply address MySQL via an internal IP (i.e. through Vagrant/Docker's private networking functionality) - but obviously if we're going to use an external server, then that would fall down.

Perhaps this is a task we need to chop up into separate parts?

• [ ] Provide an API for the build environment to get / update build data (and use it)
• [ ] Integrate functionality to support multiple build servers (and thus sub-builds) and a way to define (per-project) what build servers should be used.
• [ ] Add the ability to create and launch VMs on a build-by-build basis from which tests are run.

[tobiastom]

I personally do not care much about the last part about VMs but there is for sure some demand for it.

Except from that: I 100% agree.

Not sure where I might be able to help… :)

[camspiers]

I have worked with multi php environments in VMs before and have a chef system setup using vagrant and php:build

I would recommend not going down the route if possible. It is complicated, slow (because unless you use ppa's it isn't easy to avoid using things like php:build) and hard for people to set up.

One possible solution is to have multiple cron tasks running each running a different version of php, meaning that you leave it up to the installer of PHPCI to control what version of php is used and available and propagated through the plugin runs. This would also mean you can retain the behaviour of using the current environments php for plugin runs by default.

So instead of doing:

* * * * * /usr/bin/php /path/to/phpci/console phpci:run-builds

You optionally have the ability to do:

* * * * * /usr/bin/php5.3/php /path/to/phpci/console phpci:run-builds
* * * * * /usr/bin/php5.4/php /path/to/phpci/console phpci:run-builds

Or alternatively, you pass location of binaries to the phpci:run-builds task:

* * * * * /usr/bin/php /path/to/phpci/console phpci:run-builds -b "/path/to/php5.4/php" -b "/path/to/php5.3/php"

The key with this of course is that the plugins are run using the specified version of php.

You would probably want to make it so phpci.yml files can specify that versions they should use (if available). Which means that you either need to detect what version of php has been passed in, or alternatively have a way of specifying it.

* * * * * /usr/bin/php /path/to/phpci/console phpci:run-builds --php53 "/path/to/php5.4/php" --php54 "/path/to/php5.3/php" --php55 "/path/to/php5.5/php"

Thoughts?

[tobiastom]

I really like that. Keep things simple.

[rickard2]

I've read through the discussion and I'm also in favor of keeping it simple and just trying to isolate different PHP versions on the same box.

I'm not sure if it's up to PHPCI to provide the means to set these different versions up, maybe just provide configuration options to specify the path to the different binaries and then recommend something like phpenv to create different versions of PHP?

[rcrowe]

I think an easier (first) route to take would be to support phpenv (https://github.com/CHH/phpenv) like TravisCI does.

phpenv lets you switch versions of php easily. I propose the following syntax:

php:
  - 5.4.20
  - 5.5

If the versions isn't found (phpenv versions) then mark as skipped. With the example above, 5.5 would be used to match the latest 5.5.X version installed.

[rcrowe]

Just saw @rickard2 comment. I'm all for phpenv support

[freez10]

I'll just leave it here https://github.com/c9s/phpbrew

phpbrew builds and installs multiple version php(s) in your $HOME directory. phpbrew also manage the environment variables, so you can use, switch php version whenever you need.

What phpbrew can do for you:

• Build php with different variants like PDO, mysql, sqlite, debug ...etc.
• Compile apache php module and seperate them by different versions.
• Build and install php(s) in your home directory, so you don't need root permission.
• Switch versions very easily and is integrated with bash/zsh shell.
• Automatic feature detection.
• Install & enable php extensions into current environment with ease.
• Install multiple php into system-wide environment.

[meadsteve]

Well that's definitely worth a look. We could implement a section where you provide phpbrew statements:

5.3.10 +mysql+sqlite+cgi 5.5.1 +mysql+sqlite+cgi etc...

and then all of the test plugins get run in each version.

[tobiastom]

Please note that phpbrew seems to be only available on OS X?

Am 06.02.2014 um 13:47 schrieb Steve B notifications@github.com:

Well that's definitely worth a look. We could implement a section where you provide phpbrew statements:

5.3.10 +mysql+sqlite+cgi 5.5.1 +mysql+sqlite+cgi etc...

and then all of the test plugins get run in each version.

— Reply to this email directly or view it on GitHub.

[meadsteve]

The requirements mention Ubuntu and Debian as well. But it does mean that windows users wouldn't be able to use this. My preference would be that with configuration omitted PHPCI uses the default version of php (as it does now).

[meadsteve]

And just posting one more tool that might be useful when this gets done:

https://github.com/virtphp/virtphp

[stephenreay]

I just thought I'd chime in on this with some thoughts:

As I mentioned in #189, I believe things like this ought to be pluggable, to allow for different needs.

Personally, I wouldn't dream of trying to run CI tests on a single environment that keeps switching versions of software. For me, VMs are a must here.

Also, I want to be able to define how those VMs are created - our environment (as it is for a lot of organisations) is already virtualised (we use Xen), why would I want to add more virtualisation on top of that, when I can just spin up Xen VMs, and leave the machine controlling CI (the Master in Jenkins terminology) "clean".

As for the speed thing - Travis don't have a single base image and then when its needed, spin up an instance, and do all the installs. They have a heap of different configurations (they use Vagrant boxes) and use snapshots/rollbacks to ensure each test run starts from a "clean" state.

So ideally, PHPCI would support basically the same concept - let a plugin control how slave VMs are activated/configured (so e.g. we would have a series of pre-built images and it would either reset an existing VM or provision a new one from the required image) and then run the install stage, tests, etc as normal (except over SSH).

Yes, this makes setup more complex because there are more requirements, but it also makes it much more powerful. It shouldn't be fantastically hard to provide a basic setup that uses Vagrant and PHPCI supplied box configurations.

[shulard]

I think this feature need to be implemented with a non VM tool too... Theses tools are awesome but we just need different php version not full environments.

Maybe VMs can be added as an extra feature but I think it's not the purpose of this issue.

Like in Travis, the concept of sub build can be added, then a build must generate multiple sub build instance that will be run and reported one by one. It's a big change in the interface and codebase but I think it's the most user friendly way to manage it.

Tools like php-build allow to install versions with custom plugins/modules and then tests might be run in various configurations... In combination with virtPHP it can be easy to enable a specific environment without side effets on the system (PHPCI must not change the global php version or install a global module...)

[dancryer]

The actual implementation will let you use whatever method you so choose. We'll simply provide the ability to create multiple builds for different platforms, it's up to you how you run them.

We'll additionally provide docker based build agents for common configurations, and a method of running them, but that will be optional. On 30 Jul 2014 16:50, "Stéphane HULARD" notifications@github.com wrote:

I think this feature need to be implemented with a non VM tool too... Theses tools are awesome but we just need different php version not full environments.

Maybe VMs can be added as an extra feature but I think it's not the purpose of this issue.

Like in Travis, the concept of sub build can be added, then a build must generate multiple sub build instance that will be run and reported one by one. It's a big change in the interface and codebase but I think it's the most user friendly way to manage it.

Tools like php-build allow to install versions with custom plugins/modules and then tests might be run in various configurations...

— Reply to this email directly or view it on GitHub https://github.com/Block8/PHPCI/issues/112#issuecomment-50634385.

[huglester]

Hello,

Currently, I myself, run multiple versions using: https://github.com/phpbrew/phpbrew I just compiled serveral versions and can switch between them (cli or web).

Somehow there should be support (maybe) to add 'Multiple path to PHP' maybe?

So we have PhpVersion plugin, and then add: And then, in the phpci.yml we add something like:

phpversions:
    /usr/bin/php
    /opt/php/bin/php-55
    /opt/php/bin/php-5431

And then entire build should be run 3 times (I mean we will have three different results at the dashboard), one after another, because we will have some those php stats with images, etc. and some may or may not complete on some of PHP versions.

We would need to have some 'howto install additional versions'; I have some for PHPBrew and Debian7

Thanks huglester

[magnetik]

Docker seems to be the way to go.

Any ETA on this?

[guestisp]

Any update on this ?

[deatharse]

I thought I could get this working with phpenv and the shell plugin but was unsucessful. Possibly due to bad understanding of how the shell plugin works.

On the CLI i can run:

$ phpenv versions
* system (set by /home/user/.phpenv/version)
  5.3.29
$ phpenv global 5.3.29
$ phpenv versions
  system
* 5.3.29 (set by /home/user/.phpenv/version)

which correctly sets the php version so on my phpci.yml file i added the entries:

setup:
   shell:
      - "phpenv versions && phpenv global 5.3.29 && phpenv versions && php -v"

and:

complete:
   shell:
      - "phpenv versions && phpenv global system && phpenv versions"

however that resulted in the following build log entries:

Working copy created: /
RUNNING PLUGIN: shell
* system (set by RBENV_VERSION environment variable)
  5.3.29
* system (set by RBENV_VERSION environment variable)
  5.3.29
PHP 5.4.39-0+deb7u2 (cli) (built: Mar 25 2015 08:33:29) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
PLUGIN: SUCCES

as we can see it did not switch correctly but this may be a starting point.

[andreustimm]

Someone found a solution to this problem?

[chrisandchris]

Is this still open?