Can't deploy Citygo configurations generate by CAMP

[fermenreq]

Characteristics

• Issue Type: bug

• Reproducibility: always

• Severity: crash

• Tool/Service/Component: CAMP v0.6.2-git.e5ef976 (MIT) Copyright (C) 2017 -- 2019 SINTEF Digital Linux 3.10.0-862.11.6.el7.x86_64 (debian 9.11) Python 3.5.3.final.0 Z3 4.8.4 Docker version 19.03.2, build 6a30dfca03 docker-compose version 1.22.0, build f46880fe Linux 3.10.0-862.11.6.el7.x86_64 (debian 9.11) Python 3.5.3.final.0 Z3 4.8.4 Docker version 19.03.2, build 6a30dfca03 docker-compose version 1.22.0, build f46880fe

• Execution Environment: Linux 0718fb74527d 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 GNU/Linux

• Reporter: Fernando Mendez, fernando.mendez@atos.net

Description

Cant deploy any of CityGo configurations generated by CAMP

root@0718fb74527d:/camp/samples/stamp/atos# cat camp_execute.log

camp@bash:./out/config_1/images$ bash build_images.sh Sending build context to Docker daemon 2.048kB Sending build context to Docker daemon 57.62MB Step 1/11 : FROM camp-python_0 Sending build context to Docker daemon 2.048kB All images ready. Error response from daemon: Dockerfile parse error line 3: FROM requires either one or three arguments pull access denied for camp-python_0, repository does not exist or may require 'docker login' Error response from daemon: Dockerfile parse error line 3: FROM requires either one or three arguments

camp@bash:./out/config_1$ docker-compose up -d build path /camp/samples/stamp/atos/out/Citygo/Postgres/new either does not exist, is not accessible, or is not a valid URL.

camp@bash:./out/config_1$ docker-compose down --volumes --rmi all build path /camp/samples/stamp/atos/out/Citygo/Postgres/new either does not exist, is not accessible, or is not a valid URL.

Steps to reproduce

node("master") { stage('Pull SCM sut'){ git (credentialsId: 'git-credentials', url: 'https://gitlab.atosresearch.eu/ari/stamp_docker_citygoApp', branch: 'master') } stage('Run CAMP'){ sh ''' docker pull fchauvel/camp:dev; docker run -d --rm -i -t --name camp -v /var/run/docker.sock:/var/run/docker.sock -v /var/jenkins_home/workspace/citygo_camp/:/stamp_docker_citygoApp/ -t fchauvel/camp:dev ''' } stage('CAMP generate'){ sh ''' docker exec -d camp sh -c 'cp -R /stamp_docker_citygoApp/apache/app/* ./samples/stamp/atos/template/citygo/'; docker exec -i camp sh -c 'cd samples/stamp/atos/ && camp generate -d . --all'; exit; ''' } stage('CAMP realize'){ sh ''' docker exec -i camp sh -c 'cd samples/stamp/atos/ && camp realize -d .'; exit; ''' } stage('CAMP execute'){ sh ''' docker exec -i camp sh -c 'cd samples/stamp/atos/ && camp execute -d .'; exit; ''' } }

Other files and URLs

Camp configuration generated

root@0718fb74527d:/camp/samples/stamp/atos/out# ls config_1 config_2 config_3 root@0718fb74527d:/camp/samples/stamp/atos/out# cd config_1 root@0718fb74527d:/camp/samples/stamp/atos/out/config_1# ls configuration.dot configuration.yml docker-compose.yml images

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1# cat configuration.dot digraph Config { node [shape="record",style="filled",fillcolor="white"]; subgraph cluster_1 { label="container 1"; style="filled"; color="lightgrey"; citygo_0 [label="citygo"]; python_0 [label="python"]; ubuntu_0 [label="ubuntu"]; citygo_0 -> python_0; python_0 -> ubuntu_0; } subgraph cluster_2 { label="container 2"; style="filled"; color="lightgrey"; mongo_0 [label="mongo"]; } subgraph cluster_3 { label="container 3"; style="filled"; color="lightgrey"; browser_0 [label="browser"]; } subgraph cluster_4 { label="container 4"; style="filled"; color="lightgrey"; apache_0 [label="apache"]; } subgraph cluster_5 { label="container 5"; style="filled"; color="lightgrey"; hub_0 [label="hub"]; } subgraph cluster_6 { label="container 6"; style="filled"; color="lightgrey"; postgres_0 [label="postgres"]; } citygo_0 -> mongo_0; citygo_0 -> postgres_0; browser_0 -> hub_0; apache_0 -> citygo_0; hub_0 -> apache_0; }

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1# cat configuration.yml instances: apache_0: configuration: {} definition: apache service_providers:

• citygo_0 browser_0: configuration: {} definition: browser service_providers:
• hub_0 citygo_0: configuration: {} definition: citygo feature_provider: python_0 service_providers:
• mongo_0
• postgres_0 hub_0: configuration: {} definition: hub service_providers:
• apache_0 mongo_0: configuration: {} definition: mongo service_providers: [] postgres_0: configuration: {} definition: postgres service_providers: [] python_0: configuration: {} definition: python feature_provider: ubuntu_0 service_providers: [] ubuntu_0: configuration: {} definition: ubuntu service_providers: []

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1# cat docker-compose.yml version: '2' services: db: build: ../Citygo/Postgres/new/ container_name: my_postgresql environment:

• max_connections=500
• shared_buffers=256
• port=5432
• POSTGRES_DB=citygo_malaga
• POSTGRES_USER=citygo
• POSTGRES_PASSWORD=5X6sdoq0!?az=v2aSX
• PGDATA=/var/lib/postgresql/data/pgdata ports:
• "5432:5432" volumes:
• postgres_data:/var/lib/postgresql/data/ expose:
• "5432" networks:
• my-app-bridge apache: build: . container_name: my_apache environment:
• HTTPD_VERSION=2.4
• StartServers=2
• MinSpareThreads=25
• MaxSpareThreads=75
• ThreadLimit=64
• ThreadsPerChild=25
• MaxRequestWorkers=150
• MaxConnectionsPerChild=0
• DJANGO_SETTINGS_MODULE=citygo_settings.settings
• BROWSERNAME=chrome
• PLATFORM=LINUX
• JAVASCRIPTENABLED=True
• MAXINSTANCES=50
• CSSSELECTORSENABLED=True
• BROWSERCONNECTIONENABLED=True volumes:

  - ../Citygo/www:/var/www/html

• ./app/:/usr/src/app/
• static_volume:/usr/src/app/static ports:
• "80:80" depends_on:
• db
• mongo networks:
• my-app-bridge web: build: ../dashboard container_name: "my_web" restart: always environment:
• DJANGO_SETTINGS_MODULE=citygo_settings.settings
• BROWSERNAME=chrome
• PLATFORM=LINUX
• JAVASCRIPTENABLED=True
• MAXINSTANCES=50
• CSSSELECTORSENABLED=True
• BROWSERCONNECTIONENABLED=True networks:
• my-app-bridge volumes:
• ./app/:/usr/src/app/
• static_volume:/usr/src/app/static depends_on:
• db
• mongo
• apache ports:
• "8000:8000" hub: container_name: "selenium_hub" image: selenium/hub volumes:
• /dev/shm:/dev/shm/ ports:
• "4444:4444" networks:
• my-app-bridge selenium_chrome: container_name: selenium_chrome image: selenium/node-chrome-debug environment:
• HUB_PORT_4444_TCP_ADDR=hub
• HUB_PORT_4444_TCP_PORT=4444 ports:
• "5900:5900" depends_on:
• hub networks:
• my-app-bridge mongo: container_name: "my_mongo" image: mongo:4.0.6-xenial environment:
• MONGO_DATA_DIR=/data/db
• MONGO_LOG_DIR=/dev/null volumes:
• ./data/db:/data/db ports:
• "27017:27017" expose:
• "27017" networks:
• my-app-bridge volumes: postgres_data: static_volume: networks: my-app-bridge: driver: bridge

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1# cat images/apache_0/ Dockerfile demo_site.conf mpm_event.conf

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1#` cat images/apache_0/Dockerfile

FROM httpd:2.4

MAINTAINER Fernando Mendez Requena fernando.mendez@atos.net

RUN echo "sudo usermod -a -G user apache"

RUN apt-get update -y && \ apt-get upgrade -y && \ apt-get install -y \ libapache2-mod-wsgi \ python2.7 \ python-pip \ lynx \ apache2

RUN pip install mod_wsgi RUN pip install virtualenv ADD ./requirements.txt ./ RUN pip install -r requirements.txt

WORKDIR ./ ADD ./mpm_event.conf ./

RUN mkdir -p /etc/httpd/sites-enabled

RUN mkdir -p /etc/httpd/sites-available

ADD ./demo_site.conf /etc/apache2/sites-available/000-default.conf

ADD ./httpd.conf /usr/local/apache2/conf/httpd.conf

ADD ./httpd.conf /etc/httpd/conf/httpd.conf

CMD service apache2 relaod RUN chmod 777 -R /var/log/apache2 RUN cd /etc/apache2/mods-enabled/ && \ echo "envsubst < $pwd/mpm_event.conf > ./mpm_event.conf" EXPOSE 80 3500

CMD ["apache2ctl", "-D", "FOREGROUND"]

root@0718fb74527d:/camp/samples/stamp/atos/out/config_1/images# cat build_images.sh ... docker build -t camp-python_0 ./python_0 docker build -t camp-citygo_0 ./citygo_0 docker build -t camp-python_0 ./python_0 echo 'All images ready.'

Relationships

Citygo

Help on issue template

Preview to follow the link or open file .github/ISSUE_DOC.md

[fchauvel]

Hello,

Thanks for the feedback.

From the traces you've uploaded, I suspect there are several issues going on here.

1. We must ensure that the docker-compose file are consistent with the template structure and the camp.yml. I see for instance build: ../Citygo/Postgres/new, which I guess is wrong because CAMP generate configuration into /out/config_1/ (thought I don't know the directory layout you are using).
2. There may as well be an issue with the script file that CAMP generates. I will double check this and come back you you.

[fermenreq]

Hello @fchauvel ,

please have a look this PR # d0faee60340d9c065e924a166b790173397adb92 which changes in postgres path instance and some fix to lowercases. Could you please update CAMP docker image tagged as dev to Test JMetter test over CI/CD jenkins ?

root@osboxes:/home/osboxes/Desktop/STAMP/camp/samples/stamp/atos# tree -L 3

├── camp.yml ├── Citygo │   ├── apache │   │   ├── demo_site.conf │   │   ├── docker-compose.yml │   │   ├── Dockerfile │   │   ├── mpm_event.conf │   │   └── requirements.txt │   ├── app │   │   └── backend │   ├── nginx │   │   ├── docker-compose.yml │   │   └── nginx │   ├── Postgres │   │   ├── Dockerfile │   │   ├── init-db.sql │   │   └── postgresql.conf │   └── Tests │   └── citygo_tests ├── README.md └── template ├── apache │   ├── demo_site.conf │   ├── Dockerfile │   └── mpm_event.conf ├── browser │   ├── Dockerfile │   └── tests.py ├── citygo │   └── Dockerfile ├── docker-compose-apache-as-proxy.yml ├── docker-compose-apache.yml ├── docker-compose-nginx.yml ├── nginx │   ├── Dockerfile │   └── nginx.tmpl ├── postgres │   ├── Dockerfile │   ├── init-db.sql │   └── postgresql.conf └── python └── Dockerfile

16 directories, 26 files