datamohikaan commented 4 years ago

macos-cat:waardepapieren boscp08$ . bfg.bash mm 4 waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

install

container brings itś own file system, on every type of system

step	description	command
1	install docker download	`sudo install -y docker docker-common docker-client`
2	enable docker daemon	`systemctl enable docker`
3	and start docker daemon	`systemctl start docker`
4	verify that docker daemon is active by running your first container	`docker run hello-world`

docker configuration

but now as a 'normal' user

step	description
1	`groupadd docker`
2	`usermod -aG docker boscp08`
3	`systemctl restart docker`
4	`docker run hello-world`

hello from docker This message shows that your installation appears to be working correctly.

cleanup docker images and containers

step	description	command
1	remove alle containers	`docker stop $(docker ps -a -q)`
2	remove alle containers and images	`docker rm $(docker ps -a -q) && docker rmi $(docker images -q)`
3	remove all stopped containers (just waist of storage)	`docker container prune`

docker stats --no-stream

#!/bin/bash
#Open Docker, only if is not running
if (! docker stats --no-stream ); then
  # On Mac OS this would be the terminal command to launch Docker
  open /Applications/Docker.app
 #Wait until Docker daemon is running and has completed initialisation
while (! docker stats --no-stream ); do
  # Docker takes a few seconds to initialize
  echo "Waiting for Docker to launch..."
  sleep 1
done
fi

get docker-cleanup.bash

curl -o cleanup.bash https://raw.githubusercontent.com/boschpeter/my-simple-pipeline/master/cleanup.bash

git clone master branch

git clone https://github.com/discipl/waardepapieren.git

get bfg.bash

curl -o bfg.bash https://raw.githubusercontent.com/boschpeter/waardepapieren/master/bfg.bash

FQDN setter https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

ezahr LPC

sudo nano etc/hosts

127.0.0.1       localhost
#85.159.97.71    ubuntu
#127.0.0.1       discipl.westeurope.azurecontainer.io
#127.0.0.1       discipl.westeurope.azurecontainer.io
#127.0.0.1        waardepapieren.westeurope.cloudapp.azure.com
127.0.0.1         waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

255.255.255.255 broadcasthost
::1             localhost
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
# End of section

***  Welcome to a Darwin docker build  20200304_11_54 
***
***  bfg.bash   
***  /Users/boscp08/waardepapieren  
***  to build new waardepapieren images and containers 
***  FQDN = https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl 
***  docker-tag = 4.0
***  AZURE ACI-resourcegroup=Discipl_Wigo4it_DockerGroup4.0
***  LOGFILE=/Users/boscp08/waardepapieren/LOG_DIR/LOG_20200304_11_54.log
***
#######################
## variables
#######################
LOG_DIR=/Users/boscp08/waardepapieren/LOG_DIR
GIT_USER=boschpeter
GIT_REPO=waardepapieren
GITHUB_DIR=/Users/boscp08/waardepapieren
DOCKER_USER=boscp08
COMPOSE_BUILD_FLAG= --build
MOCK_NLX_IMAGE=waardepapieren_mock-nlx
WAARDEPAPIEREN_SERVICE_IMAGES=waardepapieren_waardepapieren-service
CLERK_FRONTEND_IMAGE=waardepapieren_clerk-frontend
DOCKER_VERSION_TAG=4.0
AZURE_USER=bosch.peter@outlook.com
AZ_RESOURCE_GROUP=Discipl_Wigo4it_DockerGroup4.0
AZ_DNSNAMELABEL=waardepapieren
TIMEZONE=
EPHEMERAL_RETENTION_TIME_COMPOSE_TRAVIS=2592020
CERT_HOST_IP=waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl
CERT_HOST_IP_WP_SERVICE_HOSTNAME=waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

Press enter to Continue

/Users/boscp08/waardepapieren/docker-compose-travis yml

=========
enter inspect : docker-compose-travis yml 
folder        = waardepapieren 
directory     = /Users/boscp08/waardepapieren 
repo          = /Users/boscp08/waardepapieren 
function      = set_docker_compose_travis_yml_without_volumes 
=========

version: '3'
services:
  waardepapieren-service:
    volumes:
 #     - ./waardepapieren-service/system-test/certs:/certs:ro
 #     - ./waardepapieren-service/system-test/ephemeral-certs:/ephemeral-certs:ro
      - ./waardepapieren-service/configuration/:/app/configuration:ro
    build: waardepapieren-service/.
    links:
      - mock-nlx
    ports:
      - 3232:3232
      - 3233:3233
    environment:
      - WAARDEPAPIEREN_CONFIG=/app/configuration/waardepapieren-config-compose-travis.json
      # Ignore self-signed ephemeral cert issues
      - NODE_TLS_REJECT_UNAUTHORIZED=0
  clerk-frontend:
    build:
      context: clerk-frontend/
      args:
        - CERTIFICATE_HOST=http://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:8880
    links:
      - waardepapieren-service
    ports:
      - 443:443
      - 8880:8880
    healthcheck:
      test: service nginx status
#    volumes:
#      - ./clerk-frontend/nginx/certs:/etc/nginx/certs:ro
  mock-nlx:
    build: mock-nlx/
    ports:
      - 80:80

=========
enter inspect : docker-compose-travis yml 
folder        = waardepapieren 
directory     = /Users/boscp08/waardepapieren 
repo          = /Users/boscp08/waardepapieren 
function      = set_docker_compose_travis_yml_without_volumes 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/mock-nlx/Dockerfile

=========
enter inspect : Dockerfile 
folder        = mock-nlx 
directory     = /Users/boscp08/waardepapieren/mock-nlx 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_mock_nlx_without_volumes 
=========

FROM node:10
RUN mkdir /app
ADD index.js package.json package-lock.json /app/
WORKDIR /app
RUN npm install --production
CMD npm start

=========
enter inspect : Dockerfile 
folder        = mock-nlx 
directory     = /Users/boscp08/waardepapieren/mock-nlx 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_mock_nlx_without_volumes 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/clerk-frontend/Dockerfile

=========
enter inspect : Dockerfile 
folder        = clerk-frontend 
directory     = /Users/boscp08/waardepapieren/clerk-frontend 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_clerk_frontend_without_volumes 
=========

FROM node:10
RUN mkdir /app
ADD package.json package-lock.json /app/
ENV REACT_APP_EPHEMERAL_ENDPOINT=https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443/api/eph
ENV REACT_APP_EPHEMERAL_WEBSOCKET_ENDPOINT=wss://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443/api/eph-ws
WORKDIR /app
RUN npm install --unsafe-perm
ADD public /app/public
ADD src /app/src
ARG CERTIFICATE_HOST
ENV REACT_APP_CERTIFICATE_HOST=http://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:8880
RUN npm run build
FROM nginx:1.15.8
ADD nginx/nginx.conf /etc/nginx/nginx.conf
COPY --from=0 /app/build /usr/share/nginx/html
RUN apt-get update && apt-get install -y iputils-ping
ENV TZ=Europe/Amsterdam
RUN ln -snf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime && echo Europe/Amsterdam > /etc/timezone
ENV REACT_APP_EPHEMERAL_ENDPOINT=https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443/api/eph
ENV REACT_APP_EPHEMERAL_WEBSOCKET_ENDPOINT=wss://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443/api/eph-ws
RUN mkdir /etc/nginx/certs
ADD nginx/certs/org.crt /etc/nginx/certs/org.crt
ADD nginx/certs/org.key /etc/nginx/certs/org.key

=========
enter inspect : Dockerfile 
folder        = clerk-frontend 
directory     = /Users/boscp08/waardepapieren/clerk-frontend 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_clerk_frontend_without_volumes 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/waardepapieren-service/Dockerfile

=========
enter inspect : Dockerfile 
folder        = waardepapieren-service 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_waardepapieren_service_without_volumes 
=========

FROM node:10
RUN mkdir /app
ADD .babelrc package.json package-lock.json /app/
ADD src/* app/src/
ADD configuration/* app/configuration/
#- ./waardepapieren-service/system-test/certs:/certs:ro
ENV WAARDEPAPIEREN_CONFIG=/app/configuration/waardepapieren-config-compose.json
RUN mkdir /certs
ADD system-test/certs/org.crt /certs/org.crt
ADD system-test/certs/org.key /certs/org.key
#- ./waardepapieren-service/system-test/ephemeral-certs:/ephemeral-certs:ro
RUN mkdir /ephemeral-certs
ADD system-test/ephemeral-certs/org.crt /ephemeral-certs/
ADD system-test/ephemeral-certs/org.key /ephemeral-certs/
#- ./waardepapieren-service/configuration/:/app/configuration:ro
WORKDIR /app
RUN mkdir /configuration
ADD configuration/waardepapieren-config-compose-travis.json /app/configuration
ADD configuration/waardepapieren-config-compose.json /app/configuration
ADD configuration/waardepapieren-config.json /app/configuration
ENV WAARDEPAPIEREN_CONFIG /app/configuration/waardepapieren-config.json
RUN npm install --production
CMD npm start

=========
enter inspect : Dockerfile 
folder        = waardepapieren-service 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service 
repo          = /Users/boscp08/waardepapieren 
function      = set_Dockerfile_waardepapieren_service_without_volumes 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/clerk-frontend/nginx/nginx conf

=========
enter inspect : nginx conf 
folder        = nginx 
directory     = /Users/boscp08/waardepapieren/clerk-frontend/nginx 
repo          = /Users/boscp08/waardepapieren 
function      = set_clerk_frontend_nginx_conf 
=========

events {
    worker_connections  1024;
}

http {

    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }

    # Http server to obtain NLX certificate
    server {
        listen 8880;

        location / {
           root /usr/share/nginx/html;
           include /etc/nginx/mime.types;
        }
    }

    server {
        listen 443 ssl;

        ssl_certificate /etc/nginx/certs/org.crt;
        ssl_certificate_key /etc/nginx/certs/org.key;

        location /api/eph/ {
            proxy_pass https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:3232/;
        }

        location /api/eph-ws {
            proxy_pass https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:3232;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection Upgrade;
        }
        location / {
            root /usr/share/nginx/html;
            include /etc/nginx/mime.types;
        }
    }
}

=========
enter inspect : nginx conf 
folder        = nginx 
directory     = /Users/boscp08/waardepapieren/clerk-frontend/nginx 
repo          = /Users/boscp08/waardepapieren 
function      = set_clerk_frontend_nginx_conf 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/config-compose-travis.json

=========
enter inspect : waardepapieren-config-compose-travis.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_compose_travis_json 
=========

{
  "EPHEMERAL_ENDPOINT" : "https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:3232",
  "EPHEMERAL_WEBSOCKET_ENDPOINT" : "wss://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:3232",
  "EPHEMERAL_CERT": "/ephemeral-certs/org.crt",
  "EPHEMERAL_KEY": "/ephemeral-certs/org.key",
  "NLX_OUTWAY_ENDPOINT" : "http://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:80",
  "NLX_CERT": "/certs/org.crt",
  "NLX_KEY": "/certs/org.key",
  "LOG_LEVEL": "info",
  "EPHEMERAL_RETENTION_TIME": 86400,
  "PRODUCT_NEED" : "BRP_UITTREKSEL_NEED",
  "SOURCE_NLX_PATH" : "/brp/basisregistratie/natuurlijke_personen/bsn/{BSN}",
  "SOURCE_ARGUMENT" : "BSN",
  "PRODUCT_ACCEPT" : "BRP_UITTREKSEL_ACCEPT",
  "PRODUCT_NAME" : "Gewaarmerkt digitaal afschrift van gegevens uit de basisregistratie personen (BRP)",
  "PRODUCT_DESCRIPTION" : "Uittreksel Basis Registratie Persoonsgegevens",
  "PRODUCT_PURPOSE" : "Bewijs verblijfadres in woonplaats",
  "SOURCE_DATA_SELECTION" : [
    {"Burgerservicenummer (BSN)" : "burgerservicenummer"},
    {"Woonplaats verblijfadres" : "verblijfadres.woonplaats"}
  ]
}

=========
enter inspect : waardepapieren-config-compose-travis.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_compose_travis_json 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/waardepapieren-config-compose.json

=========
enter inspect : waardepapieren-config-compose.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_compose_json 
=========

{
  "EPHEMERAL_ENDPOINT" : "https://localhost:3232",
  "EPHEMERAL_WEBSOCKET_ENDPOINT" : "wss://localhost:3232",
  "EPHEMERAL_CERT": "/ephemeral-certs/org.crt",
  "EPHEMERAL_KEY": "/ephemeral-certs/org.key",
  "NLX_OUTWAY_ENDPOINT" : "https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443",
  "NLX_CERT": "/certs/org.crt",
  "NLX_KEY": "/certs/org.key",
  "LOG_LEVEL": "info",
  "EPHEMERAL_RETENTION_TIME": 86400,
  "PRODUCT_NEED" : "BRP_UITTREKSEL_NEED",
  "SOURCE_NLX_PATH" : "/brp/basisregistratie/natuurlijke_personen/bsn/{BSN}",
  "SOURCE_ARGUMENT" : "BSN",
  "PRODUCT_ACCEPT" : "BRP_UITTREKSEL_ACCEPT",
  "PRODUCT_NAME" : "Gewaarmerkt digitaal afschrift van gegevens uit de basisregistratie personen (BRP)",
  "PRODUCT_DESCRIPTION" : "Uittreksel Basis Registratie Persoonsgegevens",
  "PRODUCT_PURPOSE" : "Bewijs verblijfadres in woonplaats",
  "SOURCE_DATA_SELECTION" : [
    {"Burgerservicenummer (BSN)" : "burgerservicenummer"},
    {"Woonplaats verblijfadres" : "verblijfadres.woonplaats"}
  ]
}

=========
enter inspect : waardepapieren-config-compose.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_compose_json 
=========

Press enter to Continue

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/waardepapieren-config.json

=========
enter inspect : waardepapieren-config.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_json 
=========

{
  "EPHEMERAL_ENDPOINT" : "https://localhost:3232",
  "EPHEMERAL_WEBSOCKET_ENDPOINT" : "wss://localhost:3232",
  "EPHEMERAL_CERT": "/ephemeral-certs/org.crt",
  "EPHEMERAL_KEY": "/ephemeral-certs/org.key",
  "NLX_OUTWAY_ENDPOINT" : "http://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:80",
  "NLX_CERT": "/certs/org.crt",
  "NLX_KEY": "/certs/org.key",
  "LOG_LEVEL": "info",
  "EPHEMERAL_RETENTION_TIME": 86401,
  "PRODUCT_NEED" : "BRP_UITTREKSEL_NEED",
  "SOURCE_NLX_PATH" : "/brp/basisregistratie/natuurlijke_personen/bsn/{BSN}",
  "SOURCE_ARGUMENT" : "BSN",
  "PRODUCT_ACCEPT" : "BRP_UITTREKSEL_ACCEPT",
  "PRODUCT_NAME" : "Gewaarmerkt digitaal afschrift van gegevens uit de basisregistratie personen (BRP)",
  "PRODUCT_DESCRIPTION" : "Uittreksel Basis Registratie Persoonsgegevens",
  "PRODUCT_PURPOSE" : "Bewijs verblijfadres in woonplaats",
  "SOURCE_DATA_SELECTION" : [
    {"Burgerservicenummer (BSN)" : "burgerservicenummer"},
    {"Woonplaats verblijfadres" : "verblijfadres.woonplaats"}
  ]
}

=========
enter inspect : waardepapieren-config.json 
folder        = configuration 
directory     = /Users/boscp08/waardepapieren/waardepapieren-service/configuration 
repo          = /Users/boscp08/waardepapieren 
function      = set_waardepapieren_service_config_json 
=========

Press enter to Continue

docker-compose -f docker-compose-travis.yml up. 15'

macos-cat:~ boscp08$ docker stats --no-stream
CONTAINER ID        NAME                                      CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
b276244dfadb        waardepapieren_clerk-frontend_1           0.00%               2.211MiB / 1.943GiB   0.11%               1.18kB / 189B       0B / 0B             2
6a5c36c42011        waardepapieren_waardepapieren-service_1   0.00%               72.57MiB / 1.943GiB   3.65%               1.92kB / 609B       19.6MB / 4.1kB      24
8b324af57d2a        waardepapieren_mock-nlx_1                 0.00%               34.02MiB / 1.943GiB   1.71%               1.54kB / 0B         3.36MB / 4.1kB      20

docker commit tag and push 4.0

macos-cat:waardepapieren boscp08$ .bfg mm 4.0  waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

 M A I N - M E N U 
~~~~~~~~~~~~~~~~~~~~~
https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443  DOCKER-VERSION_TAG=4.0 
~~~~~~~~~~~~~~~~~~~~~
10. docker_system_prune                                     
11. get_curl_waardepapieren                                 
12  show_parameters                                         
20. set_all_Dockerfiles   waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl                     
21. set_docker_compose_travis_yml_without_volumes           
22. set_Dockerfile_mock_nlx                                 
23. set_Dockerfile_clerk_frontend_without_volumes           
24. set_Dockerfile_waardepapieren_service_without_volumes   
25. set_clerk_frontend_nginx_conf                           
26. set_waardepapieren_service_config_compose_travis_json    
27. set_azure_deploy_aci_yaml            waardepapieren           
30. docker_compose_images                 --build waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend  
31. docker_compose_down                  waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend  
~~~~~~~~~~~~~~~~~~~~~
40. docker_build_images                  waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend VERSION=4.0
41. docker_build_mock_nlx                waardepapieren_mock-nlx with DOCKER_VERSION_TAG=4.0 
42. docker_build_waardepapieren_service  waardepapieren_waardepapieren-service with DOCKER_VERSION_TAG=4.0 
43. docker_build_clerk_frontend          waardepapieren_clerk-frontend with DOCKER_VERSION_TAG=4.0 
44. docker_login                         boscp08               
46. docker_commit_containers                 waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend TAG=4.0 
49. docker_push_images                   waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend TAG=4.0 
~~~~~~~~~~~~~~~~~~~~~
50. azure_restart_ACI                    Discipl_Wigo4it_DockerGroup4.0         
51. azure_login                          bosch.peter@outlook.com                
52. azure_delete_resourcegroup           Discipl_Wigo4it_DockerGroup4.0         
53. azure_create_resourcegroup           Discipl_Wigo4it_DockerGroup4.0         
54. azure_create_ACI          Azure Container Instance costcenter;-) = Discipl_Wigo4it_DockerGroup4.0         
~~~~~~~~~~~~~~~~~~~~~
60. https://github.com/boschpeter/waardepapieren   
61. https://hub.docker.com/?ref=login         
62. https://portal.azure.com/\#home           
63. https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl:443                 
64. pim https://waardepapieren-demo.discipl.org BSN=663678651
~~~~~~~~~~~~~~~~~~~~~
70.  . bfg.bash mm 0 localhost 
71.  . bfg.bash mm 1 waardepapieren-demo.westeurope.cloudapp.azure.com 
72.  . bfg.bash mm 2 discipl.westeurope.cloudapp.azure.com
73.  . bfg.bash mm 3 waardepapieren-demo.westeurope.azurecontainer.io
74.  . bfg.bash mm 4 discipl.westeurope.azurecontainer.io
79.    get_this_bfg_batchfile_generator latest from https://github.com/boschpeter/waardepapieren.git 
~~~~~~~~~~~~~~~~~~~~~
80. the_whole_sjebang 
81. 
82. 
~~~~~~~~~~~~~~~~~~~~~
99. Exit
Enter choice [ 1 - 99] 44

docker login

macos-cat:waardepapieren boscp08$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: boscp08
Password: P....!20..
Login Succeeded

46. docker_commit_containers                 waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend TAG=4.0 
49. docker_push_images                   waardepapieren_mock-nlx + waardepapieren_waardepapieren-service + waardepapieren_clerk-frontend TAG=4.0

docker commit

boscp08@hp:~/waardepapieren$ echo  ${DOCKER_USER}  ${GIT_REPO} ${MOCK_NLX} ${DOCKER_VERSION_TAG}
boscp08 waardepapieren mock-nlx 4 0
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${MOCK_NLX} ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren mock-nlx 4 0
sha256:be030d8bc81202d360789ab20a4976a628da89aa3efe492f0ae1fc078cca36bb
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${WAARDEPAPIEREN_SERVICE}  ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren waardepapieren-service 4 0
sha256:817233b9000f1498d0bd4c708529679749de7ba50f9e9e271e6687f86728b01f
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${CLERK_FRONTEND} ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren clerk-frontend 4 0
sha256:8ecf2d84fd5705d7a36fb66c5ed03bb74e006a54485695650eaff3c737e52ce1

boscp08@hp:~/waardepapieren$ echo  ${DOCKER_USER}  ${GIT_REPO} ${MOCK_NLX} ${DOCKER_VERSION_TAG}
boscp08 waardepapieren mock-nlx 4 0
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${MOCK_NLX} ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren mock-nlx 4 0
sha256:be030d8bc81202d360789ab20a4976a628da89aa3efe492f0ae1fc078cca36bb
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${WAARDEPAPIEREN_SERVICE}  ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren waardepapieren-service 4 0
sha256:817233b9000f1498d0bd4c708529679749de7ba50f9e9e271e6687f86728b01f
boscp08@hp:~/waardepapieren$ docker_commit  ${DOCKER_USER}  ${GIT_REPO} ${CLERK_FRONTEND} ${DOCKER_VERSION_TAG}
Running:docker_commit boscp08 waardepapieren clerk-frontend 4 0
sha256:8ecf2d84fd5705d7a36fb66c5ed03bb74e006a54485695650eaff3c737e52ce1

docker images

boscp08@hp:~/waardepapieren$ docker images
REPOSITORY                                      TAG                 IMAGE ID            CREATED              SIZE
boscp08/waardepapieren_clerk-frontend           4                   8ecf2d84fd57        About a minute ago   136MB
boscp08/waardepapieren_waardepapieren-service   4                   817233b9000f        About a minute ago   979MB
boscp08/waardepapieren_mock-nlx                 4                   be030d8bc812        About a minute ago   913MB
waardepapieren_clerk-frontend                   latest              3e979793718c        27 minutes ago       136MB
<none>                                          <none>              df80384d1bd7        28 minutes ago       1.62GB
waardepapieren_waardepapieren-service           latest              9e903bed3b6d        34 minutes ago       979MB
waardepapieren_mock-nlx                         latest              5d9ab6e8de07        35 minutes ago       913MB
node                                            10                  aa6432763c11        7 days ago           911MB
nginx                                           1.15.8              f09fe80eb0e7        13 months ago        109MB

docker push

boscp08@hp:~/waardepapieren$ docker_push_image  ${DOCKER_USER} ${GIT_REPO}_${MOCK_NLX} ${DOCKER_VERSION_TAG}
Running:docker_push_image boscp08 waardepapieren_mock-nlx 4 0
The push refers to repository [docker.io/boscp08/waardepapieren_mock-nlx]
6db96b7d1bcf: Pushed 
9b8e8e3d6590: Pushed 
54d65f775cb0: Pushed 
0f95d4bba9b0: Pushed 
cea8dd7dcda1: Mounted from library/node 
90aa9e20811b: Mounted from library/node 
92086f81cd8d: Mounted from library/node 
236d3097407d: Mounted from library/node 
91daf9fc6311: Mounted from library/node 
162804eaaa1e: Mounted from library/node 
d040e6423b7a: Mounted from library/node 
00adafc8e77b: Mounted from library/node 
2c995a2087c1: Mounted from library/node 
4: digest: sha256:9d01715df930bd9874a4282d263ef52809b6b1cf67aca09aca11cbd76a7d5587 size: 3048
boscp08@hp:~/waardepapieren$ docker_push_image  ${DOCKER_USER} ${GIT_REPO}_${WAARDEPAPIEREN_SERVICE} ${DOCKER_VERSION_TAG}
Running:docker_push_image boscp08 waardepapieren_waardepapieren-service 4 0
The push refers to repository [docker.io/boscp08/waardepapieren_waardepapieren-service]
f5f149273996: Pushed 
4de81fafba4e: Pushed 
3233e14b1859: Pushed 
6e852cc12b41: Pushed 
38699bcf2fea: Pushed 
4d38a2793873: Pushed 
a1ee727379a7: Pushed 
f722f7fe52cd: Pushed 
e0c003db1f10: Pushed 
31c1d6148218: Pushed 
c12ae5886a68: Pushed 
c4311b27c8e2: Pushed 
3fbb0aa7f1e7: Pushed 
6ac0c5b57c8a: Pushed 
fad312aa34f4: Pushed 
0f95d4bba9b0: Mounted from boscp08/waardepapieren_mock-nlx 
cea8dd7dcda1: Mounted from boscp08/waardepapieren_mock-nlx 
90aa9e20811b: Mounted from boscp08/waardepapieren_mock-nlx 
92086f81cd8d: Mounted from boscp08/waardepapieren_mock-nlx 
236d3097407d: Mounted from boscp08/waardepapieren_mock-nlx 
91daf9fc6311: Mounted from boscp08/waardepapieren_mock-nlx 
162804eaaa1e: Mounted from boscp08/waardepapieren_mock-nlx 
d040e6423b7a: Mounted from boscp08/waardepapieren_mock-nlx 
00adafc8e77b: Mounted from boscp08/waardepapieren_mock-nlx 
2c995a2087c1: Mounted from boscp08/waardepapieren_mock-nlx 
4: digest: sha256:c0f6a7d5e488e1193915896c8f6bf39b38f9df6a249b79e0dd2d6450bfad9f98 size: 5542
boscp08@hp:~/waardepapieren$ docker_push_image  ${DOCKER_USER} ${GIT_REPO}_${CLERK_FRONTEND} ${DOCKER_VERSION_TAG}
Running:docker_push_image boscp08 waardepapieren_clerk-frontend 4 0
The push refers to repository [docker.io/boscp08/waardepapieren_clerk-frontend]
776a825eaca1: Pushed 
b0323da95e01: Pushed 
417df54fd43d: Pushed 
5264a57901f6: Pushed 
9bdf7bfc7254: Pushed 
72fe14667996: Pushed 
ca5d1d5525ea: Pushed 
a0201ea9b4d7: Pushed 
6b5e2ed60418: Layer already exists 
92c15149e23b: Layer already exists 
0a07e81f5da3: Layer already exists 
4: digest: sha256:bc7efc808b9d5ab23d2a8f8326b96ded2ed42776b692303dd03ee113f4fdcedb size: 2615

Toegang tot logius Private Cloud

kubelogin

macos-cat:ictu-waardepapieren boscp08$ kubelogin
IMPORTANT NOTICE:
The credential plugin mode is available since v1.14.0.
Kubectl will automatically run kubelogin and you do not need to run kubelogin explicitly.

You can switch to the credential plugin mode by setting the following user to
/Users/boscp08/.kube/config.
---
users:
- name: oidc
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: kubectl
      args:
      - oidc-login
      - get-token
      - --oidc-issuer-url=https://keycloak.l12m.nl/auth/realms/LPC
      - --oidc-client-id=lpc-ot2-k8s
      - --oidc-client-secret=LPgQaRfVPfRmvIsPnege
---
See https://github.com/int128/kubelogin for more.

You already have a valid token until 2020-03-04 17:41:12 +0100 CET

kubectl cluster-info -n hack-ontwikkel

macos-cat:~ boscp08$ kubectl cluster-info -n hack-ontwikkel
Kubernetes master is running at https://api.lpc-ot2.k8s.easi

kubectl apply -f ./k8s/deployment.yaml

macos-cat:ictu-waardepapieren boscp08$ kubectl apply -f ./k8s/deployment.yaml
deployment.apps/waardepapieren configured
service/waardepapieren-service unchanged
ingress.extensions/waardpapier-ingress unchanged

deployment.yaml

git clone https://gitlab.l12m.nl/logius/hack/ictu-waardepapieren

https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl https://ezahr.hack-ontwikkel.lpc-ot2.l12m.nl

apiVersion: apps/v1
kind: Deployment
metadata:
  name: waardepapieren
  namespace: hack-ontwikkel
spec:
  replicas: 1
  selector:
    matchLabels:
      app: waardepapieren
  template:
    metadata:
      labels:
        app: waardepapieren
        ingress-frontoffice-policy: allow
    spec:
      containers:
        - name: waardepapieren-clerk
          image: boscp08/waardepapieren_clerk-frontend:4
          imagePullPolicy: Always
          ports:
          - containerPort: 443
        - name: waardepapieren-service
          image: boscp08/waardepapieren_waardepapieren-service:4
          imagePullPolicy: Always
          ports:
          - containerPort: 3232
        - name: waardepapieren-mock-nlx
          image: boscp08/waardepapieren_mock-nlx:4
          imagePullPolicy: Always
          ports:
          - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: waardepapieren-service
  namespace: hack-ontwikkel
spec:
  ports:
  - port: 8443
    protocol: TCP
    targetPort: 443
  selector:
    app: waardepapieren
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.allow-http: "true"
    kubernetes.io/ingress.class: nginx
  name: waardpapier-ingress

kubectl get events -n hack-ontwikkel

macos-cat:k8s boscp08$ kubectl get events -n hack-ontwikkel
LAST SEEN   TYPE      REASON              OBJECT                                 MESSAGE
3m43s       Normal    Scheduled           pod/waardepapieren-595bc779bd-4jzmh    Successfully assigned hack-ontwikkel/waardepapieren-595bc779bd-4jzmh to worker12
3m41s       Normal    Pulling             pod/waardepapieren-595bc779bd-4jzmh    Pulling image "boscp08/waardepapieren_clerk-frontend:4"
3m35s       Normal    Pulled              pod/waardepapieren-595bc779bd-4jzmh    Successfully pulled image "boscp08/waardepapieren_clerk-frontend:4"
3m24s       Normal    Created             pod/waardepapieren-595bc779bd-4jzmh    Created container waardepapieren-clerk
3m24s       Normal    Started             pod/waardepapieren-595bc779bd-4jzmh    Started container waardepapieren-clerk
3m24s       Normal    Pulling             pod/waardepapieren-595bc779bd-4jzmh    Pulling image "boscp08/waardepapieren_waardepapieren-service:4"
89s         Normal    Pulled              pod/waardepapieren-595bc779bd-4jzmh    Successfully pulled image "boscp08/waardepapieren_waardepapieren-service:4"
69s         Normal    Created             pod/waardepapieren-595bc779bd-4jzmh    Created container waardepapieren-service
69s         Normal    Started             pod/waardepapieren-595bc779bd-4jzmh    Started container waardepapieren-service
69s         Normal    Pulling             pod/waardepapieren-595bc779bd-4jzmh    Pulling image "boscp08/waardepapieren_mock-nlx:4"
65s         Normal    Pulled              pod/waardepapieren-595bc779bd-4jzmh    Successfully pulled image "boscp08/waardepapieren_mock-nlx:4"
61s         Normal    Created             pod/waardepapieren-595bc779bd-4jzmh    Created container waardepapieren-mock-nlx
61s         Normal    Started             pod/waardepapieren-595bc779bd-4jzmh    Started container waardepapieren-mock-nlx
3m43s       Normal    SuccessfulCreate    replicaset/waardepapieren-595bc779bd   Created pod: waardepapieren-595bc779bd-4jzmh


macos-cat:ictu-waardepapieren boscp08$ kubectl get pods  -n hack-ontwikkel
NAME                                READY   STATUS    RESTARTS   AGE
ezahr-769df97889-6km4t              1/1     Running   0          2d2h
ictu-vvdlaar-c8bcd7bfb-lctmb        1/1     Running   0          14d
marktplaats-6c5468c9d7-d5tbl        1/1     Running   0          79d
marktplaats-fe-cfbcdd5db-2mdft      1/1     Running   1          78d
my-test-9d8bc58cc-5brsf             1/1     Running   1          76d
nginx-deployment-5754944d6c-t9z5m   1/1     Running   0          20d
nginx-deployment-5754944d6c-vwqvf   1/1     Running   0          14d
pim-my-test-7d87cdbd6-qzhbv         1/1     Running   0          14d
rv-69fdf84d76-55tdh                 1/1     Running   0          14d
rv-69fdf84d76-vd4zf                 1/1     Running   0          14d
testproject-b4c8766d6-cg68d         1/1     Running   0          78d
waardepapieren-595bc779bd-4jzmh     3/3     Running   0          66m

azure deploy

=========
enter inspect : deploy-aci yaml 
folder        = waardepapieren 
directory     = /Users/boscp08/waardepapieren 
repo          = /Users/boscp08/waardepapieren 
function      = set_azure_deploy_aci_yaml 
=========

location: westeurope
name: Discipl_Wigo4it_DockerGroup4.0
properties:
  containers:
  - name: mock-nlx
    properties:
      image: boscp08/waardepapieren_mock-nlx:4.0
      resources:
        requests:
          cpu: 1
          memoryInGb: 0.5
      ports:
      - port: 80
  - name: waardepapieren-service
    properties:
      image: boscp08/waardepapieren_waardepapieren-service:4.0
      resources:
        requests:
          cpu: 1
          memoryInGb: 0.5
      ports:
      - port: 3232
      - port: 3233
  - name: clerk-frontend
    properties:
      image: boscp08/waardepapieren_clerk-frontend:4.0
      resources:
        requests:
          cpu: 1
          memoryInGb: 0.5
      ports:
      - port: 443
      - port: 8880
  osType: Linux
  ipAddress:
    type: Public
    # fqdn wordt: discipl_waardepapieren.westeurope.azurecontainer.io
    dnsNameLabel: waardepapieren
    ports:
    - protocol: tcp
      port: '443'
    - protocol: tcp
      port: '3232'
    - protocol: tcp
      port: '3233'
    - protocol: tcp
      port: '80'
    - protocol: tcp
      port: '8880'
tags: null
type: Microsoft.ContainerInstance/containerGroups

=========
enter inspect : deploy-aci yaml 
folder        = waardepapieren 
directory     = /Users/boscp08/waardepapieren 
repo          = /Users/boscp08/waardepapieren 
function      = set_azure_deploy_aci_yaml 
=========

datamohikaan commented 4 years ago

Basically there are two issues using docker-compose. (on a VM <> ContainerInstance)

1 docker-compose uses volumes: connects to the VM filesystem

above example copies org.key and cert.key direct into the container(image) before it is pushed to docker-hub. This is not desired, i know.!

Disclaimer just try to set up a Kubernetes build pipeline LPC and ACI Container Instances

https://hub.docker.com/repository/docker/boscp08/waardepapieren_clerk-frontend https://hub.docker.com/repository/docker/boscp08/waardepapieren_waardepapieren-service https://hub.docker.com/repository/docker/boscp08/waardepapieren_mock-nlx

Pim Otte created Branch: feature/certs-as-environment-variables for this i will try to get this working tomorrow.

2 docker-compose creates a bridged network for https://mock-nlx and https://waardepapieren-service . on-premise i applied the FQDN for https://mock-nls and and https://waardepapieren-service

grep -ilr 'https://mock-nlx' | xargs -I@ sed -i '' 's/https:\/\/mock-nlx/https:\/\/zuidugchelen.eu/g' @ grep -ilr 'https://waardepapieren-service' | xargs -I@ sed -i '' 's/https:\/\/waardepapieren-service/https:\/\/zuidugchelen.eu/g' @

this works on https://zuidugchelen.eu / Don't change localhost to FQDN thismakes the application instable after some time. (after 5 till 30 minutes)

proces described in following youtube https://www.youtube.com/watch?v=5irsAdKoEBU&list=PLLasX02E8BPCrIhFrc_ZiINhbRkYMKdPT&index=6

by 18-12-2019 this was working on ACI today almost on LPC (self certificate issue is pending)

datamohikaan commented 4 years ago

20200211 status update Below

https://gitlab.l12m.nl/logius/hack/ezahr/blob/master/.gitlab-ci.yml this pipeline shows the next possible and hopefully final steps.

1 Build docker images

  https://github.com/discipl/waardepapieren/tree/feature/certs-as-environment-variables

with https://github.com/discipl/waardepapieren/blob/feature/certs-as-environment-variables/prepare-env.sh

2 push 3 images to them to harbor repo

  2 scenario's here 
   2.1 tag image  prefererred way, but needs additional deployment config 
   2.2 commit containers to new image. NB org.key and cert.keys are build in.

3 deploy to kubernetes cluster

  see below

Build / deploy

#
# GitLab Build Pipeline
#
# CI* and KUBE* variables are defined by GitLab Settings
# 
#
image: docker:stable

stages:
  - build
  - deploy

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "" # Workaround for docker in docker failure 'Cannot connect to the Docker daemon'
  REGISTRY_SERVER: "harbor-ot.l12m.nl"
  REGISTRY_PROJECT: "hack"
  IMAGE_NAME: "ezahr-service"  # Change this 
  TAG: "0.2" # Fixed version for testing purposes by others
  DOCKER_HOST: "tcp://localhost:2375"
  CD_NAMESPACE: hack-ontwikkel

services:
  - docker:dind

before_script:
  - echo "Started by:${GITLAB_USER_LOGIN} with configuration:"
  - echo CI_PIPELINE ${CI_PIPELINE}
  - echo CI_COMMIT_SHA ${CI_COMMIT_SHA}
  - echo TAG ${TAG}
  - echo KUBE_URL ${KUBE_URL}
  - echo REGISTRY_SERVER ${REGISTRY_SERVER}
  - echo REGISTRY_PROJECT ${REGISTRY_PROJECT}
  - echo IMAGE_NAME ${IMAGE_NAME}
  - echo ROBOT_NAME ${HARBOR_ROBOT_NAME}
  - echo ROBOT_SECRET ${HARBOR_ROBOT_SECRET}
  - echo "Start $(date)"

after_script:
  - echo "End $(date)"

build-image:
  stage: build
  script:
    # Login to Registry > https://docs.gitlab.com/ee/user/project/deploy_tokens/index.html#gitlab-deploy-token
    # Hide password > https://docs.docker.com/engine/reference/commandline/login/
    - docker build -t "${REGISTRY_SERVER}/${REGISTRY_PROJECT}/${IMAGE_NAME}:${TAG}" --build-arg TAG="${CI_PIPELINE_ID}" --network=host . # Build latest; host network speeds-up remote retrievals
    - echo -n "${HARBOR_ROBOT_SECRET}" | base64 -d | docker login -u ${HARBOR_ROBOT_NAME} --password-stdin ${REGISTRY_SERVER}
    - docker push "${REGISTRY_SERVER}/${REGISTRY_PROJECT}/${IMAGE_NAME}:${TAG}"

deploy-ot2:
  stage: deploy
  environment: lpc-ot2
  image: registry.gitlab.com/quattro-public/quattro-k8s-kubectl
  dependencies: [] # Workaround > https://gitlab.com/gitlab-org/gitlab-ee/issues/6144
  only:
    - master
  script:
    # Deploy the pod in k8s
    - kubectl config set-context $(kubectl config current-context) --namespace=${CD_NAMESPACE}
    - kubectl apply -f ./k8s/deployment.yaml # Pod deployment
    - kubectl apply -f ./k8s/service.yaml # service deployment
    - kubectl apply -f ./k8s/ingress.yaml # ingress deployment

Example pull from harbor https://gitlab.l12m.nl/logius/hack/ezahr/blob/master/k8s/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ezahr # Deployment name
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ezahr
  template:
    metadata:
      labels:
        app: ezahr
        ingress-frontoffice-policy: allow
    spec:
      containers:
        - name: ezahr
          image: harbor-ot.l12m.nl/hack/ezahr-service:0.2
          imagePullPolicy: Always
          ports:
          - containerPort: 8888
          resources:
            requests:
              memory: 50Mi # 50 Megabytes
              cpu: 100m # 1 CPU = 1000 > 300m is 300/1000 = 10% CPU
            limits:
              memory: 50Mi
              cpu: 100m
      imagePullSecrets: # https://kubernetes.io/docs/concepts/configuration/secret/#using-imagepullsecrets
      - name: harbor-puller

Network proxy issue ahead ... (pim to the rescue)

https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl/ -> renders 503 Service Temporarily Unavailable nginx/1.17.8 NB this is ingress from LPC not clerkfrontend - 1.15

I proceed with above approach.

hope the run will be okay

discipl / waardepapieren

Publish docker image for waardepapieren-service #84

install

container brings itś own file system, on every type of system

docker configuration

cleanup docker images and containers

docker stats --no-stream

get docker-cleanup.bash

git clone master branch

get bfg.bash

FQDN setter https://waardepapieren.hack-ontwikkel.lpc-ot2.l12m.nl

sudo nano etc/hosts

/Users/boscp08/waardepapieren/docker-compose-travis yml

/Users/boscp08/waardepapieren/mock-nlx/Dockerfile

/Users/boscp08/waardepapieren/clerk-frontend/Dockerfile

/Users/boscp08/waardepapieren/waardepapieren-service/Dockerfile

/Users/boscp08/waardepapieren/clerk-frontend/nginx/nginx conf

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/config-compose-travis.json

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/waardepapieren-config-compose.json

/Users/boscp08/waardepapieren/waardepapieren-service/configuration/waardepapieren-config.json

docker-compose -f docker-compose-travis.yml up. 15'

docker commit tag and push 4.0

docker login

docker commit

docker images

docker push

Toegang tot logius Private Cloud

kubelogin

kubectl cluster-info -n hack-ontwikkel

kubectl apply -f ./k8s/deployment.yaml

deployment.yaml

kubectl get events -n hack-ontwikkel

azure deploy

1 docker-compose uses volumes: connects to the VM filesystem

2 docker-compose creates a bridged network for https://mock-nlx and https://waardepapieren-service . on-premise i applied the FQDN for https://mock-nls and and https://waardepapieren-service

1 Build docker images

2 push 3 images to them to harbor repo

3 deploy to kubernetes cluster