Use the newly created omegaUp Puppet module for Docker deployment

[lhchavez]

This unifies both the Docker and Vagrant provisioning to have a single configuration.

[lhchavez]

@alanboy esto se ve prometedor. Logré iniciar Docker usando el deployment de Puppet.

Caveats:

• El grader no funciona. Aún no estoy seguro, pero posiblemente sea buena idea tenerlo en un contenedor aparte. Tengo que pensar cómo implementar compartir los archivos necesarios (submissions, inputs, outputs).
• Minijail no se puede instalar. Hay problemas con las dependencias y el script mágico que uso.
• Los archivos de la base de datos están repetidos. Hay que ingeniárnosla para copiarlos del módulo de Puppet :/
• Acceder al sistema de archivos es difícil. Intenté usar VOLUME (por ejemplo, para los logs), pero los permisos están raros. Usar esto para trabajar con el repositorio desde afuera se ve que va a estar complicado para usar Docker para hacer desarrollo diario, y no me fascina la idea de agregar un servidor de SSH para poder editar las cosas desde adentro.
• Está corriendo como root. Eso suena peligroso.

[lhchavez]

Listo, todo containerizado, separado, con logging y archivos persistentes. Sigue siendo complicado para hacer desarrollo, pero ya empieza a tomar forma como ambiente de staging. Esta es la versión final de este pull request, cualquier cambio (que no sea bugfix/review fix) se hará en un pull request posterior. @alanboy pls review.

Known issues:

• Los passwords siguen hardcoded :/
• Sigue ejecutándose todo como root :/
• Minijail no funciona bien, todo da CE :/

[alanboy]

No me jalo: docker_omegaup_grader_1 exited with code 1

alan-VirtualBox docker # docker-compose up Pulling ambassador (cpuguy83/docker-grand-ambassador:latest)... Pulling repository docker.io/cpuguy83/docker-grand-ambassador c635b1699f78: Download complete 511136ea3c5a: Download complete 16386e29a1f4: Download complete 835c4d274060: Download complete 22c23ce0a90c: Download complete b03b60b200e6: Download complete d3436d137b4d: Download complete a9b37a182a35: Download complete 79a8a788466d: Download complete 3317fcfac735: Download complete 3e54d00a4c92: Download complete e5f108015860: Download complete 0a466963dbdd: Download complete ef701a3b8b5a: Download complete f3ef2eea95a4: Download complete 91c63f0f01f4: Download complete 5ac8a4aa73d5: Download complete Status: Downloaded newer image for cpuguy83/docker-grand-ambassador:latest Creating docker_ambassador_1... Creating docker_omegaup_db_1... Creating docker_omegaup_grader_1... Creating docker_omegaup_frontend_1... Creating docker_omegaup_runner_1... Attaching to docker_ambassador_1, docker_omegaup_db_1, docker_omegaup_grader_1, docker_omegaup_frontend_1, docker_omegaup_runner_1 ambassador_1 | time="2015-10-04T21:37:24Z" level=info msg="Container docker_omegaup_grader_1 does not exist or is not running, skipping for now" ambassador_1 | time="2015-10-04T21:37:24Z" level=info msg="Container docker_omegaup_runner_1 does not exist or is not running, skipping for now" ambassador_1 | time="2015-10-04T21:37:24Z" level=info msg="Container docker_omegaup_frontend_1 does not exist or is not running, skipping for now" ambassador_1 | time="2015-10-04T21:37:29Z" level=info msg="Starting proxy for docker_omegaup_grader_1" ambassador_1 | time="2015-10-04T21:37:33Z" level=info msg="Starting proxy for docker_omegaup_frontend_1" omegaup_db_1 | Initializing database omegaup_db_1 | 2015-10-04T21:37:31.809882Z 0 [Warning] InnoDB: New log files created, LSN=45790 omegaup_db_1 | 2015-10-04T21:37:32.727587Z 0 [Warning] InnoDB: Creating foreign key constraint system tables. omegaup_db_1 | 2015-10-04T21:37:32.923936Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 1ef4ad80-6ae0-11e5-969d-0242ac110013. omegaup_db_1 | 2015-10-04T21:37:32.941500Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened. omegaup_db_1 | 2015-10-04T21:37:32.954159Z 0 [Warning] Failed to setup SSL omegaup_db_1 | 2015-10-04T21:37:32.955660Z 0 [Warning] SSL error: SSL context is not usable without certificate and private key omegaup_db_1 | 2015-10-04T21:37:32.956884Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option. ambassador_1 | time="2015-10-04T21:37:35Z" level=info msg="Starting proxy for docker_omegaup_runner_1" omegaup_frontend_1 | 2015-10-04 21:37:37,748 CRIT Supervisor running as root (no user in config file) omegaup_frontend_1 | 2015-10-04 21:37:37,957 INFO supervisord started with pid 1 omegaup_frontend_1 | 2015-10-04 21:37:39,067 INFO spawned: 'nginx' with pid 15 omegaup_frontend_1 | 2015-10-04 21:37:39,408 INFO spawned: 'hhvm' with pid 16 omegaup_frontend_1 | 2015-10-04 21:37:40,429 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) omegaup_frontend_1 | 2015-10-04 21:37:40,441 INFO success: hhvm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) omegaup_db_1 | Database initialized omegaup_db_1 | MySQL init process in progress... omegaup_db_1 | MySQL init process in progress... omegaup_db_1 | MySQL init process in progress... omegaup_db_1 | MySQL init process in progress... omegaup_db_1 | 2015-10-04T21:37:48.606990Z 0 [Note] mysqld (mysqld 5.7.8-rc) starting as process 38 ... omegaup_db_1 | MySQL init process in progress... omegaup_db_1 | 2015-10-04T21:37:48.967263Z 0 [Note] InnoDB: PUNCH HOLE support available omegaup_db_1 | 2015-10-04T21:37:48.973241Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins omegaup_db_1 | 2015-10-04T21:37:48.973502Z 0 [Note] InnoDB: Uses event mutexes omegaup_db_1 | 2015-10-04T21:37:48.974364Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier omegaup_db_1 | 2015-10-04T21:37:48.974487Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 omegaup_db_1 | 2015-10-04T21:37:48.974606Z 0 [Note] InnoDB: Using Linux native AIO omegaup_db_1 | 2015-10-04T21:37:48.997107Z 0 [Note] InnoDB: Number of pools: 1 omegaup_db_1 | 2015-10-04T21:37:49.015066Z 0 [Note] InnoDB: Not using CPU crc32 instructions omegaup_db_1 | 2015-10-04T21:37:49.715760Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M ambassador_1 | time="2015-10-04T21:37:50Z" level=info msg="Stopping proxy for docker_omegaup_grader_1" ambassador_1 | time="2015-10-04T21:37:50Z" level=info msg="Stopping proxy on tcp/[::]:21680 for tcp/172.17.0.20:21680 (accept tcp [::]:21680: use of closed network connection)" docker_omegaup_grader_1 exited with code 1 Gracefully stopping... (press Ctrl+C again to force) Stopping docker_omegaup_runner_1... done Stopping docker_omegaup_frontend_1... done Stopping docker_omegaup_db_1... done Stopping docker_ambassador_1... done Exception in thread Thread-2 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/code/build/docker-compose/out00-PYZ.pyz/threading", line 810, in __bootstrap_inner File "/code/build/docker-compose/out00-PYZ.pyz/threading", line 763, in run File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.multiplexer", line 41, in _enqueue_output File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.log_printer", line 62, in _make_log_generator File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 163, in wait File "/code/build/docker-compose/out00-PYZ.pyz/docker.utils.decorators", line 20, in wrapped File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 848, in wait File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 83, in _post File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 508, in post File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 465, in request File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 573, in send File "/code/build/docker-compose/out00-PYZ.pyz/requests.adapters", line 414, in send <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error'

[lhchavez]

Había una dependencia con un cambio de master. Ya debe funcionar. Si sigue sin funcionar, intenta eliminar las imágenes y hacer docker-compose build --no-cache.

[lhchavez]

Ya le doy merge?

[alanboy]

Ya no lo probe desde la ultima vez. En la noche lo intento.

[alanboy]

No lo alance a hacer hoy. Si jala, merge.

-Alan

On Mon, Nov 16, 2015 at 6:53 PM, lhchavez notifications@github.com wrote:

Ya le doy merge?

— Reply to this email directly or view it on GitHub https://github.com/omegaup/deploy/pull/2#issuecomment-157252070.

[alanboy]

C:\Users\alanb\Code\deploy>vagrant up Bringing machine 'default' up with 'virtualbox' provider... ==> default: Box 'omegaup-vivid' could not be found. Attempting to find and install... default: Box Provider: virtualbox default: Box Version: >= 0 ==> default: Box file was not detected as metadata. Adding it directly... ==> default: Adding box 'omegaup-vivid' (v0) for provider: virtualbox default: Downloading: http://cloud-images.ubuntu.com/vagrant/vivid/current/vivid-server-cloudimg-amd64-vagrant-disk1.box default: Progress: 100% (Rate: 1971k/s, Estimated time remaining: --:--:--) ==> default: Successfully added box 'omegaup-vivid' (v0) for 'virtualbox'! ==> default: Importing base box 'omegaup-vivid'... ==> default: Matching MAC address for NAT networking... ==> default: Setting the name of the VM: deploy_default_1447928778397_6310 ==> default: Clearing any previously set forwarded ports... ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 80 => 8080 (adapter 1) default: 21680 => 21680 (adapter 1) default: 22 => 2222 (adapter 1) ==> default: Running 'pre-boot' VM customizations... ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... default: Warning: Connection timeout. Retrying... Timed out while waiting for the machine to boot. This means that Vagrant was unable to communicate with the guest machine within the configured ("config.vm.boot_timeout" value) time period.

[alanboy]

aun no investigo esto mas bien tiene que ver con mi ambiente (Windows) :(

[lhchavez]

Hmm hasta ahorita me di cuenta que estabas intentando hacer vagrant up en vez de probar Docker.

En otras noticias, estuve usando esto para hacer unas pruebas del grader-proxy. Ya definitivamente está listo para consumo normal.

[lhchavez]

Pues le voy a dar commit porque ocupo fixes de esta rama en master.