Open nazar-pc opened 9 years ago
DanielG
commented
9 years ago PHP isn't supported yet vm/language-install.sh needs to setup all the language specific stuff.
I did have a look and I think I got it to work, just waiting for the cookbooks to finish building all the supported php versions. I'll nudge you to test it later.
nazar-pc
commented
9 years ago Great, I'll be happy to help with testing.
DanielG
commented
9 years ago Looks like this is going to take a fair bit of work. The basics are working, the setup goes through but is painfully slow (building 3 different php version takes on the order of 1-2 hours). The problem is that right now the docker containers aren't setup to start daemons inside the container which the php environment needs for mysql and stuff to come up. So I'd either have to hack around that with a per-language init script or just start ubunut's /sbin/init but that has a metric shit ton of problems so I'll probably just figure out how to use something like https://github.com/phusion/baseimage-docker instead.
All modifications I made are in master and I uploaded the images to the docker registry if you want to give it a shot. I probably won't come around to fixing the init problems any time soon though since I don't have a pressing need for that myself you're welcome to try it yourself though, apart from the extremely slow build-run-debug cycle it shouldn't be too hard. You probably just have to replace ubuntu:precise with baseimage-docker's image and see if the daemons come up properly then.
nazar-pc
commented
9 years ago Well, here is my .travis.yml: https://github.com/nazar-pc/CleverStyle-CMS/blob/master/.travis.yml
PHP seems to work fine, but now I've stumbled upon:
$ sudo service memcached start memcached: unrecognized service
and
$ mysql -e 'CREATE DATABASE
cscms.travis;' ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
How to add them inside?
Tried both ubuntu:presice and phusion/baseimage, result doesn't seem to change anyhow.
I'm not familiar with all this things yet, but what is the difficulty to add all supported languages/services from the beginning and just fix bugs if someone encounter them? Build time of base image? Because it is really awesome what you're doing in this project, and it would be great to see if it works out of the box with everything that is supported by travis-ci.org
DanielG
commented
9 years ago From: Nazar Mokrynskyi notifications@github.com Subject: Re: [travis-run] The command "phpenv global 5.5" failed and exited with 127 during . (#5) Date: Mon, 24 Nov 2014 17:48:17 -0800
Well, here is my
.travis.yml: https://github.com/nazar-pc/CleverStyle-CMS/blob/master/.travis.yml PHP seems to work fine, but now I've stumbled upon:$ sudo service memcached start memcached: unrecognized service
Ahhh I have to execute all the stuff from travis' base image template in base-install.sh too (https://github.com/travis-ci/travis-images/blob/master/templates/worker.standard.yml). I forgot that existed ^^ Looks like memcached is one of the things they install in the 'standard' image.
$ mysql -e 'CREATE DATABASE
cscms.travis;' ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
Yeah that's what I got too
Tried both
ubuntu:presiceandphusion/baseimage, result doesn't seem to change anyhow.
Looks like phusion/baseimage installs a completely seperate init system which doesn't integrate with ubuntu's upstart (gah) so I don't know how to get the init stuff working yet either some google fu is in order I suppose.
I'm not familiar with all this things yet, but what is the difficulty to add all supported languages/services from the beginning and just fix bugs if someone encounter them?
That is certainly fesible I just couldn't be bothered yet to use travis' vm templates1 directly instead I'm encoding this manually in language-install.sh2 as that was Good Enough^{TM} for what I was using it for.
Build time of base image?
Well the build environment has to come from somewhere :) travis-run uses travis' chef cookbooks to re-recreate the same environment they're using.
To do this I use 2+n (where n is the number of supported languages) docker images:
script (see docker/Dockerfile.script)
sets up a container with travis-build/travis-core and all of it's dependencies installed, this is needed to generate the build scripts from the .travis.yml
This is a separate container since packaging all the stuff travis-core depends on is infesible and probably too much of a moving targer anyways. Also I didn't want travis-run to depend on ruby ;)
vm/base-install.sh just installs a few ubuntu packages
and chef-solovm/language-install.sh uses chef-solo and travis' cookbooks to
do all the language specific setupUnfortunately travis builds almost everything important in their
cookbooks completely from scratch while only using something like
make -j 1 which makes everything painfully slow but downloading all
the source/packages also takes ages and I have to start that from
scratch every time I change anything in the base image :/
I append the travis-run version used to generate the images to the docker image tag name resulting in stuff like:
base_v0.2.3 script_v0.2.3-1-gc44ba70 // this is mostly just the output of git describe
I then push those images to docker's cloudy stuff so you / other users
don't have to rebuild this stuff from scratch every time they use
travis-run create.
Because it is really awesome what you're doing in this project, and it would be great to see if it works out of the box with everything that is supported by travis-ci.org
Thanks, yeah I'd like that. Right now the largest problem is the impedance mismatch between the virtualization travis is actually using in production (VirtualBox of all things) and linux containers. There seems to be some work going on to move their virtualization to docker/lxc though at which point it will become infinitly easier to achieve this goal though it will probably need a complete rewrite.
nazar-pc
commented
9 years ago BTW, why don't you use docker automatic image building? It may ease deployment of new images for you a lot since you don't have to build image on local machine.
DanielG
commented
9 years ago BTW, why don't you use docker automatic image building? It may ease deployment of new images for you a lot since you don't have to build image on local machine.
I have scripts that automate the build so there isn't really any easing left to do :p. I run the build on a VPS so the upload of the built image doesn't take forever (~10min for the last run).
But I have to test build the images before I do that either way ;) so whether I'm waiting for docker's servers to build the image or my local machine / VPS doesn't really matter.
There are also a few other reasons why I don't do this: travis-run just doesn't fit in docker's narrow minded model of how images should be built.
I don't have a single dockerfile for a the repo but rather have a bunch of dockerfiles + some that are generated by the shell scripts.
I tried to keep travis-run independent of the underlying virtualization layer which makes conforming with docker's arbitrary conventions somewhat inconvinent to impossible.
I started out with schroot then switched over to vagrant for platform
independence, then that annoyed me and I switched to docker (while the
other backends don't work anymore I'm considering supporting qemu at
some point in the future) so I'd like to keep in independent of the
virtualization. Right now all the docker specifics are constrained to
docker/ and the scripts in vm/ could be run inside any VM to get a
travis environment.
DanielG
commented
9 years ago @nazar-pc I'm making progress towards making this work after all. I found a Dockerfile that works around ubuntu's upstart problems which I'm now using as the basis for the travis-run images. I also automated generating the chef-solo runlists from travis' worker templates. I'm building all the images right now to give it a spin, will probably take a few hours/days since they support quite a lot of languages ;)
nazar-pc
commented
9 years ago Well, I've just tried to run everything and error occurred during travis-run create:
-----------------
| BUILD ERROR |
-----------------
Here are the last 10 lines from the log:
-----------------------------------------
/tmp/php-build/source/xdebug-2.2.6/xdebug_var.c:769:5: warning: passing argument 2 of 'zend_get_object_classname' from incompatible pointer type [enabled by default]
/home/travis//.phpenv/versions/5.2.17/include/php/Zend/zend_API.h:266:14: note: expected 'char **' but argument is of type 'const char **'
/tmp/php-build/source/xdebug-2.2.6/xdebug_var.c: In function 'xdebug_var_synopsis_text_ansi':
/tmp/php-build/source/xdebug-2.2.6/xdebug_var.c:873:4: warning: passing argument 2 of 'zend_get_object_classname' from incompatible pointer type [enabled by default]
/home/travis//.phpenv/versions/5.2.17/include/php/Zend/zend_API.h:266:14: note: expected 'char **' but argument is of type 'const char **'
/tmp/php-build/source/xdebug-2.2.6/xdebug_var.c: In function 'xdebug_var_export_xml_node':
/tmp/php-build/source/xdebug-2.2.6/xdebug_var.c:1296:4: warning: passing argument 2 of 'zend_get_object_classname' from incompatible pointer type [enabled by default]
/home/travis//.phpenv/versions/5.2.17/include/php/Zend/zend_API.h:266:14: note: expected 'char **' but argument is of type 'const char **'
shtool:echo:Warning: unable to determine terminal sequence for bold mode
shtool:echo:Warning: unable to determine terminal sequence for bold mode
-----------------------------------------To be honest, I have no idea why PHP 5.2 is here, it is not supported anymore, just like 5.3 too. I'm running tests for 5.4+, is it possible to avoid building of PHP 5.3- at all?
About automatic builds - I'd prefer automatic builds because there is more trust, there is some guarantee about what is inside container since it is build from public source code directly from repository.
Tried to run existing .travis.yml that works fine with Travis and got:
I'm worried mainly about:
But also:
Looks weird, Docker normally doesn't allow to edit
/etc/hosts. Also filescomponents/modules/PhpMyAdmin/pma/config.header.inc.phpandincludes/js/ab.uikit-2.11.0.min.jswere removed, first 2 days ago, first - even more, it is obvious that they are not exists anymore (many other files were removed too, but only this two causes error), how travis-run found them?