GenweiWu / Blog

个人技术能力提升
MIT License
4 stars 0 forks source link

docker #37

Open GenweiWu opened 6 years ago

GenweiWu commented 6 years ago

入门

https://docs.docker.com/get-started/

镜像和容器

镜像是一个软件包,包括了软件本身以及软件运行所需的环境

容器则是由镜像启动的,即启动一个镜像得到一个容器,就是一个镜像的实例 docker ps用来查看当前机器上的容器

docker pull

~/.docker/config.json

"proxies":
{
"default":
{
"httpProxy": "http://127.0.0.1:3001",
"httpsProxy": "http://127.0.0.1:3001",
"noProxy": "*.test.example.com,.example2.com"
}
}
  1. 重启docker
    systemctl daemon-reload
    systemctl restart docker
GenweiWu commented 6 years ago

基本信息

//docker信息
docker info

清理

https://docs.docker.com/config/pruning/

docker system prune

常用命令

查看所有镜像 
docker images
删除镜像
docker rmi xxx
docker rmi -f xxx yyy (强制删除,即使现在有容器在使用这个镜像)
查询tag为none的镜像
docker images -f "dangling=true" -q
删除所有tag为none的镜像
docker rmi -f $(docker images -f "dangling=true" -q)

查看端口映射
[root@SZX1000538971 ~]# docker port fcafa4a5b031
50000/tcp -> 0.0.0.0:50000
8080/tcp -> 0.0.0.0:8080

查看所有容器
docker ps -a

打包
docker build -t library/java-demo:jdk_8 .
docker build -t library/java-demo:jdk_8 -f dockerfile.one  .
docker build --no-cache -t library/java-demo:jdk_8 -f dockerfile.one  .

进入docker
docker exec -it 29198c060396 /bin/sh    
docker exec -it 29198c060396 bash
拷贝文件
docker cp /opt/1.zip 29198c060396:/opt
查看资源占用
docker stats
//查看日志
docker logs --tail 20  171fe3dd515d
docker logs 171fe3dd515d | less
//类似tailf一样的效果
//https://docs.docker.com/engine/reference/commandline/logs/
docker logs --follow xxx
docker logs --tail 100 --follow xxx

查看镜像的细节
docker inspect 171fe3dd515d 
重命名container
docker rename 345df9ed5b47 new_name

镜像的导出导入
//镜像导出:最好用repo:tag而不是镜像id,因为后者load的时候会丢失tag
docker save -o <path for generated tar file> <image name>
docker save -o c:/myfile.tar centos:16
#这个好像也可以
docker save centos:16 -o c:/myfile.tar 
//镜像导入
docker load -i <path to image tar file>

//通过查看sha256判断是不是同一个镜像
docker images --digests

指定用户进入容器

docker exec -it --user root <container id> /bin/bash

编写测试

https://blog.csdn.net/qq_35981283/article/details/80738451

进入容器进行命令行输入,方便测试
docker run -d -it library/java-demo:jdk_8 /bin/bash
更好的方法top会一直输出内容,就不会超时退出了
docker run -d -it library/java-demo:jdk_8 top

docker push到私有镜像

You need to tag your image correctly first with your registryhost:

docker tag [OPTIONS] IMAGE[:TAG] [REGISTRYHOST/][USERNAME/]NAME[:TAG]

Then docker push using that same tag.

docker push NAME[:TAG]

Example:

docker tag 518a41981a6a myRegistry.com/myImage
docker push myRegistry.com/myImage

注意

docker启动失败的时候,docker exec 是无法执行的

但是此时docker cp是可执行的
GenweiWu commented 6 years ago

docker远程调试

基础

对应java8:

JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
java $JAVA_OPTS -jar analyse.jar $*

针对docker(参考)

  1. docker启动命令 java $JAVA_OPTS -xxx.jar
  2. docker -e JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
GenweiWu commented 5 years ago

dockerfile入门

https://yeasy.gitbooks.io/docker_practice/image/build.html

COPY 和 ADD

因此在 COPY 和 ADD 指令中选择的时候,可以遵循这样的原则,所有的文件复制均使用 COPY 指令,仅在需要自动解压缩的场合使用 ADD。

COPY 111.txt /opt/dir1
COPY dir/222.txt  /home/222.txt

ADD的自动解压缩适用于tar.gz,zip不可以

ADD 111.zip   //发现没有被解压

ENV 设置环境变量

换行处理; 内容有空格;

ENV VERSION=1.0 DEBUG=on \
NAME="Happy Feet"

https://stackoverflow.com/questions/8633461/how-to-keep-environment-variables-when-using-sudo 注意:ENV的设置只是对root生效,使用sudo切换用户会丢失

EXPOSE 声明端口

要将 EXPOSE 和在运行时使用 -p <宿主端口>:<容器端口> 区分开来。-p,是映射宿主端口和容器端口,换句话说,就是将容器的对应端口服务公开给外界访问,而 EXPOSE 仅仅是声明容器打算使用什么端口而已,并不会自动在宿主进行端口映射。

WORKDIR 指定工作目录(当前目录)

错误的写法是这样的:并不会找到 /app/111.txt;因为两个RUN命令互相不影响,不会保留状态;

RUN cd /app
RUN echo 'hello' > 111.txt

WORKDIR可以多次设置


FROM ubuntu:14.04

RUN echo '111' > /home/111.txt RUN echo cat /home/111.txt //111

RUN echo cat 111.txt //cat: 111.txt: No such file or directory

WORKDIR /home RUN echo cat 111.txt //111

WORKDIR / RUN echo cat 111.txt //cat: 111.txt: No such file or directory

CMD echo hello


> WORKDIR是针对相对路径的,绝对路径不受影响

FROM ubuntu:14.04

WORKDIR /home RUN echo 'aaa' > a.txt \ && echo 'bbb' > /b.txt

RUN echo cat a.txt //aaa RUN echo cat b.txt //cat: b.txt: No such file or directory RUN echo cat /b.txt //bbb

WORKDIR / RUN echo cat a.txt //cat: a.txt: No such file or directory RUN echo cat /home/a.txt //aaa RUN echo cat b.txt //bbb

CMD echo hello


##  RUN 运行命令

注意:
**1. export变量需要在同一个RUN里面,下面的第一个`echo $a`是`1`,第二个是空字符串**

RUN export a=1 \ && echo $a

RUN $a


**2.RUN set -x 可以方便的进行debug出执行情况**

RUN set -x \ ls -l xxx


## ARG 用于设置环境变量,也可以人为传参数覆盖默认的
- ARG设置的变量在运行时失效的,ENV的在运行时还是有效的
- ARG可以设置默认值,可以在build时动态传递参数

```dockerfile
FROM centos
ARG NAME='Hello World'
RUN echo 'this is:'${NAME}
[root@SZX1000538971 argsTest]# docker build -t arg-test .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM centos
 ---> 0e541f96f4a3
Step 2/3 : ARG NAME='Hello World'
 ---> Using cache
 ---> aa57ff9b35fb
Step 3/3 : RUN echo 'this is:'${NAME}
 ---> Running in 591d17c58183
this is:Hello World
 ---> 3325607b8027
Removing intermediate container 591d17c58183
Successfully built 3325607b8027
Successfully tagged arg-test:latest
[root@SZX1000538971 argsTest]# docker build --build-arg NAME='Dave'  -t arg-test .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM centos
 ---> 0e541f96f4a3
Step 2/3 : ARG NAME='Hello World'
 ---> Using cache
 ---> aa57ff9b35fb
Step 3/3 : RUN echo 'this is:'${NAME}
 ---> Running in 9eca98cc8e71
this is:Dave
 ---> 33410030fdd4
Removing intermediate container 9eca98cc8e71
Successfully built 33410030fdd4
Successfully tagged arg-test:latest

多个变量的话,每个变量都要用一个build-arg

docker build \
-t essearch/ess-elasticsearch:1.7.6 \
--build-arg number_of_shards=5 \
--build-arg number_of_replicas=2 \
--no-cache .

ARG会在运行时失效


FROM centos

ARG NAME='111'

//这句在docker build时会打印出NAME的值 RUN echo '$NAME='$NAME

//这句在docker run时无法打印出NAME的值 CMD echo 'NAME='$NAME


> args可以实现动态from
```dockerfile
ARG version=2.32.2
from jenkins-base:$version

volume 进行文件夹挂载

https://segmentfault.com/a/1190000015684472

文件夹挂载

host上文件夹一定会覆盖container中文件夹:

host container mount result
文件夹不存在/文件夹存在但为空 文件夹不存在/存在但为空/存在且不为空 container中文件被覆盖(清空)
文件夹存在且不为空 文件夹不存在/存在但为空/存在且不为空 container中文件夹内容被覆盖(原内容清空, 覆盖为host上文件夹内容)

文件挂载

host上文件一定会覆盖container中文件

host container mount result
不存在的文件 已经存在的文件 禁止行为
存在的文件 不存在的文件/已经存在的文件 新增/覆盖 (若目录不存在则会创建目录)
GenweiWu commented 5 years ago

Demo

GenweiWu commented 5 years ago

docker镜像最佳实践

1. 分析docker镜像太大

docker build -t test .
docker history xxx 可以看到每一步的大小

个人总结

参考

2. 镜像开发过程中不要用-v进行挂载

3. 镜像继承可以传参数

from也可以是动态的

ARG version=3.36-1
FROM jenkins/slave:$version
docker build --build-arg version='2.0' -t test .
GenweiWu commented 5 years ago

常见问题

【问题】cannot use systemctl in dokcer container

问题: 比如安装nfs组件 https://qizhanming.com/blog/2018/08/08/how-to-install-nfs-on-centos-7

RUN yum clean all \ 
&& yum makecache \
&& yum install -y nfs-utils \
## 启动nfs服务
&& systemctl start rpcbind  //这里会报错:Failed to get D-Bus connection
&& mount -t nfs 192.168.0.101:/data /data

原因是:docker不允许在容器中运行后台服务,无法调用systemctl命令

解决方法:加上--privileged 即docker run --privileged xxx,同时dockerfile中不需要再写 systemctl start rpcbind这种代码了

【docker磁盘满了】

[root@SZX1000538971 var]# docker run -d -it gitlab /bin/bash
docker: Error response from daemon: devicemapper: Error running deviceCreate (CreateSnapDeviceRaw) dm_task_run failed.
See 'docker run --help'.

通过docker info看下:

...
Docker Root Dir: /var/lib/docker
...

然后看下磁盘空间

[root@SZX1000538971 var]# df -h /var/lib/docker/
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda8      2.9G  2.9G     0 100% /var

果然磁盘满了

解决方法

方法1:重装docker

sudo yum remove docker
sudo rm -rf /var/lib/docker
sudo yum install docker

方法2:硬重置

systemctl stop docker 
rm -rf /var/lib/docker
systemctl start docker 

【问题】docker build时无法联网

在dockerfile中 curl www.baidu.com 结果报错:curl: (7) Failed connect to 1.2.3.4:3128; No route to host

但是在宿主机环境下执行同样的命令是可以成功的。

解决方法:增加网络映射--network=host

https://stackoverflow.com/a/45751818

docker build -t my-demo .
改为
docker build --network=host -t my-demo .

对应的,docker run的命令是:

docker run -d \
    --net=host \
    -e ID=3 \
    -v /etc/localtime:/etc/localtime \
    -h 1.2.3.4 \
    --name mysql \
    mysql:5.7.19

使用宿主机网络后,-p就失效了

https://docs.docker.com/network/host/ Note: Given that the container does not have its own IP-address when using host mode networking, port-mapping does not take effect, and the -p, --publish, -P, and --publish-all option are ignored, producing a warning instead:

【问题】时区不对

ENV LC_ALL en_US.UTF-8
ENV TZ 'Asia/Shanghai'
...

【莫名其妙的错误】

docker cannot run because of "container no such file or directory

dockerfile

...
COPY start.sh /home/start
...
ENTRYPOINT ["start"]

解决方法是dos2unix start.sh再重新docker build

https://github.com/moby/moby/issues/9066#issuecomment-62875041

Problem solved, it was a line-ending problem in the sh files...

【entrypoint权限问题】

RUN yum makecache &&  yum install -y dos2unix openssl \
&& cd / \
&& dos2unix ssl_genkey.sh startup.sh \
&& chmod a+x ssl_genkey.sh startup.sh \
&& ls -l ssl_genkey.sh \
&& ls -l startup.sh

ENTRYPOINT ["/startup.sh"]

但是执行报错:

starting container process caused "exec: \"/startup.sh\": permission denied".

规避方法

https://github.com/composer/docker/issues/7#issuecomment-370086150

used
ENTRYPOINT ["sh", "/docker-entrypoint.sh"]
works for sure without using chmod

docker json.log too big

https://stackoverflow.com/q/42510002

echo "" > $(docker inspect --format='{{.LogPath}}' <container_name_or_id>)
GenweiWu commented 5 years ago

centos 系统的一些镜像

https://hub.docker.com/u/centos

docker中gosu的使用

http://yunke.science/2018/04/09/gosu-or-sudo/

GenweiWu commented 5 years ago

查看镜像内核,可以进入容器,使用如下命令查看:

cat /etc/issue  
cat /etc/os-release
cat /etc/SuSE-release 
cat /etc/redhat-release 
cat /etc/system-release 

推荐

cat /etc/os-release
GenweiWu commented 5 years ago

内存限制

docker 限制

m指总的内存限制 memory-reservation则指的内存超过memory-reservation后,则不能长期超过它,需要稳定在它限制之下

docker run 
-m 3000M 
--memory-reservation 1000M
test-service

java限制

在java命令中进行限制

-Xms1g -Xmx1g

tips

memory-reservation的限制不能保证 我试过下面的配置,发现实际上跑到2.5G也没问题...

docke run 
JAVA_OPTS='-Xms2g -Xmx2g' 
-m 3000M
--memory-reservation 1000M
GenweiWu commented 5 years ago

docker run的用法

docker run -d --privileged \
    --hostname 10.11.12.13 \
    -p 80:80 -p 443:443 -p 11006:22 \
    -v /mnt/gitlab/backup:/share_files/gitlab_backup \
    -v /gitlab/logs:/var/log/gitlab \
    -e  username="ritchie" \
    --log-opt max-size=200m \
    --log-opt max-file=5 \
    --name gitlab \
    my-gitlab:9.5.3-ce.0

-p: 指定端口映射,格式为:主机(宿主)端口:容器端口 -v: 磁盘映射 : 主机磁盘:容器磁盘 -e: 设置环境变量

GenweiWu commented 4 years ago

docker调试

覆盖CMD命令

docker exec -it $CONTAINER_ID /bin/bash

覆盖entrypoint https://serverfault.com/a/594486

docker run -dit --entrypoint=/bin/bash $IMAGE
GenweiWu commented 4 years ago

[Gist] docker run自动进入bash

dockerfile


FROM centos:7

ENV LC_ALL zh_CN.UTF-8 ENV TZ 'Asia/Shanghai'

ENTRYPOINT ["/bin/bash"]


> build 

docker build -t test .


> run

[root@SZX1000538971 test]# docker run -it test bash: warning: setlocale: LC_ALL: cannot change locale (zh_CN.UTF-8) /bin/sh: warning: setlocale: LC_ALL: cannot change locale (zh_CN.UTF-8) [root@d676daa2fb1f /]# exit

GenweiWu commented 4 years ago

[Gist]docker run执行命令后进入bash

dockerfile


FROM centos:7

ENV LC_ALL zh_CN.UTF-8 ENV TZ 'Asia/Shanghai'

COPY 111.sh /

RUN chmod a+x 111.sh \ && ls -l 111.sh

ENTRYPOINT ["sh" ,"/111.sh"]

ENTRYPOINT ["/bin/bash"]


> 111.sh
```sh
# /bin/bash

echo 'HelloWorld'
touch 111.txt

/bin/bash

build

docker build -t test .

run

docker run -it test
GenweiWu commented 4 years ago

docker -H远程访问能力

https://www.cnblogs.com/hongdada/p/11512901.html

GenweiWu commented 4 years ago

docker启动失败日志

https://stackoverflow.com/a/30970134

Ubuntu (old using upstart ) - /var/log/upstart/docker.log
Ubuntu (new using systemd ) - sudo journalctl -fu docker.service
Amazon Linux AMI - /var/log/docker
Boot2Docker - /var/log/docker.log
Debian GNU/Linux - /var/log/daemon.log
CentOS - /var/log/daemon.log | grep docker
CoreOS - journalctl -u docker.service
Fedora - journalctl -u docker.service
Red Hat Enterprise Linux Server - /var/log/messages | grep docker
OpenSuSE - journalctl -u docker.service
OSX - ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/log/d‌​ocker.log
Windows - Get-EventLog -LogName Application -Source Docker -After (Get-Date).AddMinutes(-5) | Sort-Object Time
journalctl -u docker.service | less
GenweiWu commented 4 years ago

docker-compose

docker-compose --version

查看配置信息
docker-compose config

创建并启动/停止并删除容器
docker-compose up -d
docker-compose down

停止容器
docker-compose start
docker-compose stop
GenweiWu commented 4 years ago

CMD vs entrypoint

https://blog.csdn.net/u010900754/article/details/78526443


The CMD instruction has three forms:

CMD ["executable","param1","param2"] (exec form, this is the preferred form) CMD ["param1","param2"] (as default parameters to ENTRYPOINT) CMD command param1 param2 (shell form)

ENTRYPOINT has two forms:

ENTRYPOINT ["executable", "param1", "param2"] (exec form, preferred) ENTRYPOINT command param1 param2 (shell form)



- CMD 一般用作默认命令
- entrypoint 可以传参数
- 同时包含cmd 和 entrypoint时,cmd是entrypoint的默认参数
GenweiWu commented 4 years ago

docker容器无法访问host机器本身的端口

现象

 I have a container A run with “-p 8080:80”, and a container B ( on the same host ) directly visit host-ip:8080, then log says: NO ROUTE TO HOST
但是另一台机器的容器C中是可以访问host-ip:8080

原因

https://forums.docker.com/t/no-route-to-host-network-request-from-container-to-host-ip-port-published-from-other-container/39063 这是一个docker的bug

解决方法

# ifconfig docker0
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::42:edff:fecd:f236  prefixlen 64  scopeid 0x20<link>
        ether 02:42:ed:cd:f2:36  txqueuelen 0  (Ethernet)
        RX packets 2937614  bytes 424863591 (405.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3398958  bytes 622190677 (593.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

## 修改防火墙
firewall-cmd --permanent --zone=public --add-rich-rule='rule family=ipv4 source address=172.17.0.0/16 accept' && firewall-cmd --reload
GenweiWu commented 4 years ago

清理dead状态的容器

https://stackoverflow.com/a/52119986