Container memo

[bird0401]

Images

mysql

• [x] volume mount database files for host not to edit it easily and opportunity to do it is rare
  • when using it in other host, share database via temporary container
• [x] bind mount my.cnf for host to edit it easily for changing settings of mysql and immediately reflecting it to the container
• [ ] create dump file for backup and volume it
• [x] keep pass secret by using host env

app

• [x] set external modules on requirements.txt and it make installing easy in one go by pip install -r requirements.txt

Others

• [x] write volume statement using type, src, dst instead of src:dst
• [ ] make image smaller using multi-stage build, uwsgi, etc.

[bird0401]

mysql

sudo docker volume create --name dog_breeds_by_name_db

sudo docker run --name db1 -dit --mount type=volume,src=dog_breeds_by_name_db,dst=/var/lib/mysql --mount type=bind,src=$PWD/my.cnf,dst=/etc/my.cnf -e MYSQL_ROOT_PASSWORD=$MYSQL_PASS --net scraping mysql:5.7

sudo docker exec -it db1 bash

backup db

sudo docker run --rm --mount type=volume,src=dog_breeds_by_name_db,dst=/backup_target --mount type=bind,src="$PWD",dst=/bind_dir busybox tar czf /bind_dir/backup/db_backup.tar.gz -C /backup_target .

[bird0401]

python

scraping

sudo docker run -dit --name app1 --mount type=bind,src=$PWD,dst=/usr/local/mount_dir -p 8081:80 --net scraping -w /usr/local/mount_dir python:3.10

sudo docker exec -it app1 bash

pip install -r requirements.txt 

ML

sudo docker run -dit --name ml1 --mount type=bind,src=$PWD,dst=/usr/local/mount_dir -w /usr/local/mount_dir python:3.10

[bird0401]

Network

sudo docker network create scraping

sudo docker network inspect scraping

[bird0401]

Dockerfile

https://docs.docker.jp/engine/reference/builder.html#volume https://docs.docker.jp/engine/userguide/eng-image/dockerfile_best-practice.html

mysql

sudo docker build -t my_mysql .

sudo docker run --name db2 -dit --mount type=volume,src=dog_breeds_by_name_db,dst=/var/lib/mysql --net scraping my_mysql

sudo docker exec -it db2 bash

python

sudo docker build -t my_python .

sudo docker run -dit --name app2 --mount type=bind,src=$PWD,dst=/usr/local/mount_dir --net scraping -w /usr/local/mount_dir my_python

sudo docker exec -it app2 bash

Others

• [x] make build context smallest by making folder for Dockerfile
• [x] install least packages
• [x] smallest layers and optimized layers order for utilize build cache

[bird0401]

Docker compose

• .env must exist to execute. I must scp it.

[bird0401]

Reference

• Singularity 2.5 official document

Singularity

• no bind mount because singularity automatically mounts $HOME
• backup database and copy it by scp to HPC cluster

export PATH=/home/app/singularity/bin:$PATH

singularity pull --name my_mysql.simg docker://mot1536/my_mysql
singularity pull --name my_python.simg docker://mot1536/my_python

singularity instance.start my_mysql.simg db3
singularity instance.start my_python.simg app3

singularity instance.list

singularity shell instance://db3
singularity shell instance://app3

singularity instance.stop db3
singularity instance.stop app3

[bird0401]

ML

• use google container registry to use it as base image of jupyter notebook