search

containers / ansible-podman-collections

Repository for Ansible content that can include playbooks, roles, modules, and plugins for use with the Podman tool
GNU General Public License v3.0
260 stars 141 forks source link

Inspect image of container for better idempotency #39

Closed sshnaidm closed 4 years ago

sshnaidm commented 4 years ago

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Podman container module should inspect image of container to understand better what are options will be set for container. It's required for better idempotency of podman_container module.

Steps to reproduce the issue:

  1. For example if we have options set in Dockerfile/image like workdir, volume and any other supported, it'll be non-default in container inspection. 
    - containers.podman.podman_container:
image: graphiteapp/graphite-statsd:latest
name: graphite
state: present
    - containers.podman.podman_container:
name: mysql
image: mysql
state: present
pod: telemetry
env:
  MYSQL_ROOT_PASSWORD: "2211"
  2. For strict image idempotency we'll need to check SHAs of various images.
jaudriga commented 4 years ago

Is there a recommeded workaround for this issue?

sshnaidm commented 4 years ago

@jaudriga actually it's an enhancement, that "nice to have". But idempotency with image should be fine now. Which issue do you hit with latest version?

jaudriga commented 4 years ago

@sshnaidm while idempotency with image may be fine, podman still recreates the container for the newest version of Graphite (where I set volume as well). I actually do not see a difference:

$ grep version /home/jrsr/.ansible/collections/ansible_collections/containers/podman/MANIFEST.json
  "version": "1.0.0",
$ ansible-playbook full.yml -CDv
...
TASK [graphite : Install Graphite container] *************************************************************************************************************************************************
--- before
+++ after
@@ -1,2 +1,2 @@
-image - graphite-statsd
-volume - ['/opt/graphite/conf:/opt/graphite/conf', '/opt/graphite/storage:/opt/graphite/storage', '027ddf42535de1ee53af893177e6a32170fd0c238a180e4cfea0778746bfb7f5:/opt/graphite/webapp/graphite/functions/custom', '0e5a5cf16d17f21ee38d91be6adc73d5f1ef77e3752b2f656e9ba2beeda0cfb9:/etc/logrotate.d', '3479cd25d0bc32d6f939cfeb1a8299f26a943e7b2102d1d3f3491c2654b767a4:/var/lib/redis', '5660a16982ded92a33754d10675a55ffbfb9cf801da2a6ee390ca01c0aaf8426:/etc/nginx', '9de5c5bfe685e5ae3c61487cdcb71615d36708c767a7607d67143f183cdfb334:/var/log', 'd3547bf26de911697ac434e574235703b85908603a6f208bbba21c76719b1076:/opt/statsd/config']
+image - graphite-statsd:1.1.7-2
+volume - ['/opt/graphite/conf:/opt/graphite/conf', '/opt/graphite/storage:/opt/graphite/storage']

I actually see that behavior for most of my containers (docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.9, graylog/graylog:3.3.0-1, mongo:4.0.18). Only grafana/grafana:7.0.0, seems to work well. But maybe we should stick to talking about graphite.

So, considering I am not doing anything wrong, my question is can I somehow use podman_container and set volume for graphite without it being recreated during every run?

sshnaidm commented 4 years ago

@jaudriga can you please paste your playbook for reproducing? Or at least the part with podman_container

jaudriga commented 4 years ago

@sshnaidm the following recreates the container on every run with the above diff:

- name: Install Graphite container
  containers.podman.podman_container:
    env:
      REDIS_TAGDB: 1
    image: "graphiteapp/graphite-statsd:1.1.7-2"
    name: graphite
    ports:
      - "80:80"
      - "2003-2004:2003-2004"
    state: present
    volumes:
      - "/opt/graphite/conf:/opt/graphite/conf"
      - "/opt/graphite/storage:/opt/graphite/storage"
jaudriga commented 4 years ago

@sshnaidm thanks for working on this so fast. Now.. it seems to handle locally mounted folders well. However, it does not seem to have resolved the issue with the image name:

$ ansible-playbook -CDv full.yml
TASK [graphite : Install Graphite container] *********************************************************************************************************************************************************************************************************************************************************************************************************
--- before
+++ after
@@ -1,2 +1,2 @@
-image - graphite-statsd
-volume - ['/opt/graphite/conf:/opt/graphite/conf']
+image - graphite-statsd:1.1.7-2
+volume - ['/opt/graphite/conf:/opt/graphite/conf', 'graphite-data:/opt/graphite/storage']

changed: [graphite.exp] => {"actions": ["recreated graphite"], "changed": true, "container": {"AppArmorProfile": "container-default-1.9.3", "Args": ["/entrypoint"], "BoundingCaps": ["CAP_AUDIT_WRITE", "CAP_CHOWN", "CAP_DAC_OVERRIDE", "CAP_FOWNER", "CAP_FSETID", "CAP_KILL", "CAP_MKNOD", "CAP_NET_BIND_SERVICE", "CAP_NET_RAW", "CAP_SETFCAP", "CAP_SETGID", "CAP_SETPCAP", "CAP_SETUID", "CAP_SYS_CHROOT"], "Config": {"Annotations": {"io.container.manager": "libpod", "io.kubernetes.cri-o.Created": "2020-06-02T12:28:28.279713316Z", "io.kubernetes.cri-o.TTY": "false", "io.podman.annotations.autoremove": "FALSE", "io.podman.annotations.init": "FALSE", "io.podman.annotations.privileged": "FALSE", "io.podman.annotations.publish-all": "FALSE", "org.opencontainers.image.stopSignal": "1"}, "AttachStderr": false, "AttachStdin": false, "AttachStdout": false, "Cmd": null, "CreateCommand": ["podman", "container", "run", "--name", "graphite", "--env", "REDIS_TAGDB=1", "--publish", "80:80", "--publish", "2003-2004:2003-2004", "--volume", "/opt/graphite/conf:/opt/graphite/conf", "--volume", "graphite-data:/opt/graphite/storage", "--detach=True", "graphiteapp/graphite-statsd:1.1.7-2"], "Domainname": "", "Entrypoint": "/entrypoint", "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "TERM=xterm", "HOSTNAME=530686241e65", "container=podman", "STATSD_INTERFACE=udp", "REDIS_TAGDB=1", "HOME=/root"], "Hostname": "530686241e65", "Image": "docker.io/graphiteapp/graphite-statsd:latest", "Labels": {"maintainer": "Denys Zhdanov <denis.zhdanov@gmail.com>"}, "OnBuild": null, "OpenStdin": false, "StdinOnce": false, "StopSignal": 1, "Tty": false, "User": "", "Volumes": null, "WorkingDir": "/"}, "ConmonPidFile": "/var/run/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/conmon.pid", "Created": "2020-06-02T12:28:28.279713316Z", "Dependencies": [], "Driver": "overlay", "EffectiveCaps": ["CAP_AUDIT_WRITE", "CAP_CHOWN", "CAP_DAC_OVERRIDE", "CAP_FOWNER", "CAP_FSETID", "CAP_KILL", "CAP_MKNOD", "CAP_NET_BIND_SERVICE", "CAP_NET_RAW", "CAP_SETFCAP", "CAP_SETGID", "CAP_SETPCAP", "CAP_SETUID", "CAP_SYS_CHROOT"], "ExecIDs": [], "ExitCommand": ["/usr/bin/podman", "--root", "/var/lib/containers/storage", "--runroot", "/var/run/containers/storage", "--log-level", "error", "--cgroup-manager", "systemd", "--tmpdir", "/var/run/libpod", "--runtime", "runc", "--events-backend", "file", "container", "cleanup", "530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119"], "GraphDriver": {"Data": {"LowerDir": "/var/lib/containers/storage/overlay/07eb7a8f8b30c8d9f2a2be1d3169b06b350f9e2bb87dcc4e8585ef8403e64d75/diff:/var/lib/containers/storage/overlay/4ef8d93c3f7e0fcafcb8e91640739f254030b78e780de8ae3bac8be8b1ab4008/diff:/var/lib/containers/storage/overlay/858f6f478edde3c923ec55b03da94412f35468d2e7771a594eed39bdd3b10a07/diff:/var/lib/containers/storage/overlay/beee9f30bc1f711043e78d4a2be0668955d4b761d587d6f60c2c8dc081efb203/diff", "MergedDir": "/var/lib/containers/storage/overlay/f5f8d30c6a376ae42a6505ac3bedb09a36f1020dcd12d538cf809693012ead4b/merged", "UpperDir": "/var/lib/containers/storage/overlay/f5f8d30c6a376ae42a6505ac3bedb09a36f1020dcd12d538cf809693012ead4b/diff", "WorkDir": "/var/lib/containers/storage/overlay/f5f8d30c6a376ae42a6505ac3bedb09a36f1020dcd12d538cf809693012ead4b/work"}, "Name": "overlay"}, "HostConfig": {"AutoRemove": false, "Binds": ["58f646ddf256925f996949bfa286b67e5dce61bab537495d9c760bf85fa4d619:/var/lib/redis:rprivate,rw,nodev,exec,nosuid,rbind", "504d0ecf1b68e612b62824618dd8a23f8b51e53a280a8d8e4bef22c0736b36ab:/var/log:rprivate,rw,nodev,exec,nosuid,rbind", "9db6dcc7d5734c0c1633543ce5511ccf7a501496231f1aab48c036cd96bed129:/etc/logrotate.d:rprivate,rw,nodev,exec,nosuid,rbind", "cde9a3e2e5491ea7313814f2ebc712d7ab48d7954b06ed48f6346c14aa652966:/etc/nginx:rprivate,rw,nodev,exec,nosuid,rbind", "graphite-data:/opt/graphite/storage:rw,rprivate,nosuid,nodev,rbind", "1a1cb9230b2075432cb13836164a5220e41ceb1ca1dfd5ba8808176bb7768039:/opt/graphite/webapp/graphite/functions/custom:rprivate,rw,nodev,exec,nosuid,rbind", "fd4e6a9f601231af729313e3c82a9efaee3fa0d375634cfdbc46883798e3110b:/opt/statsd/config:rprivate,rw,nodev,exec,nosuid,rbind", "/opt/graphite/conf:/opt/graphite/conf:rw,rprivate,rbind"], "BlkioDeviceReadBps": null, "BlkioDeviceReadIOps": null, "BlkioDeviceWriteBps": null, "BlkioDeviceWriteIOps": null, "BlkioWeight": 0, "BlkioWeightDevice": null, "CapAdd": [], "CapDrop": [], "Cgroup": "", "CgroupParent": "", "Cgroups": "default", "ConsoleSize": [0, 0], "ContainerIDFile": "", "CpuCount": 0, "CpuPercent": 0, "CpuPeriod": 0, "CpuQuota": 0, "CpuRealtimePeriod": 0, "CpuRealtimeRuntime": 0, "CpuShares": 0, "CpusetCpus": "", "CpusetMems": "", "Devices": [], "DiskQuota": 0, "Dns": [], "DnsOptions": [], "DnsSearch": [], "ExtraHosts": [], "GroupAdd": [], "IOMaximumBandwidth": 0, "IOMaximumIOps": 0, "IpcMode": "", "Isolation": "", "KernelMemory": 0, "Links": null, "LogConfig": {"Config": null, "Type": "k8s-file"}, "Memory": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": -1, "NanoCpus": 0, "NetworkMode": "default", "OomKillDisable": false, "OomScoreAdj": 0, "PidMode": "", "PidsLimit": 4096, "PortBindings": {"2003/tcp": [{"HostIp": "", "HostPort": "2003"}], "2004/tcp": [{"HostIp": "", "HostPort": "2004"}], "80/tcp": [{"HostIp": "", "HostPort": "80"}]}, "Privileged": false, "PublishAllPorts": false, "ReadonlyRootfs": false, "RestartPolicy": {"MaximumRetryCount": 0, "Name": ""}, "Runtime": "oci", "SecurityOpt": [], "ShmSize": 65536000, "Tmpfs": {}, "UTSMode": "", "Ulimits": [{"Hard": 1048576, "Name": "RLIMIT_NOFILE", "Soft": 1048576}, {"Hard": 32768, "Name": "RLIMIT_NPROC", "Soft": 32768}], "UsernsMode": "", "VolumeDriver": "", "VolumesFrom": null}, "HostnamePath": "/var/run/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/hostname", "HostsPath": "/var/run/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/hosts", "Id": "530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119", "Image": "0869fbfe98ecbb9438e53c9bc733fcb2e73b652d519fe04912bae0f69126abf2", "ImageName": "docker.io/graphiteapp/graphite-statsd:latest", "IsInfra": false, "LogPath": "/var/lib/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/ctr.log", "LogTag": "", "MountLabel": "", "Mounts": [{"Destination": "/var/lib/redis", "Driver": "local", "Mode": "", "Name": "58f646ddf256925f996949bfa286b67e5dce61bab537495d9c760bf85fa4d619", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/58f646ddf256925f996949bfa286b67e5dce61bab537495d9c760bf85fa4d619/_data", "Type": "volume"}, {"Destination": "/var/log", "Driver": "local", "Mode": "", "Name": "504d0ecf1b68e612b62824618dd8a23f8b51e53a280a8d8e4bef22c0736b36ab", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/504d0ecf1b68e612b62824618dd8a23f8b51e53a280a8d8e4bef22c0736b36ab/_data", "Type": "volume"}, {"Destination": "/etc/logrotate.d", "Driver": "local", "Mode": "", "Name": "9db6dcc7d5734c0c1633543ce5511ccf7a501496231f1aab48c036cd96bed129", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/9db6dcc7d5734c0c1633543ce5511ccf7a501496231f1aab48c036cd96bed129/_data", "Type": "volume"}, {"Destination": "/etc/nginx", "Driver": "local", "Mode": "", "Name": "cde9a3e2e5491ea7313814f2ebc712d7ab48d7954b06ed48f6346c14aa652966", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/cde9a3e2e5491ea7313814f2ebc712d7ab48d7954b06ed48f6346c14aa652966/_data", "Type": "volume"}, {"Destination": "/opt/graphite/storage", "Driver": "local", "Mode": "", "Name": "graphite-data", "Options": ["nosuid", "nodev", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/graphite-data/_data", "Type": "volume"}, {"Destination": "/opt/graphite/webapp/graphite/functions/custom", "Driver": "local", "Mode": "", "Name": "1a1cb9230b2075432cb13836164a5220e41ceb1ca1dfd5ba8808176bb7768039", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/1a1cb9230b2075432cb13836164a5220e41ceb1ca1dfd5ba8808176bb7768039/_data", "Type": "volume"}, {"Destination": "/opt/statsd/config", "Driver": "local", "Mode": "", "Name": "fd4e6a9f601231af729313e3c82a9efaee3fa0d375634cfdbc46883798e3110b", "Options": ["nodev", "exec", "nosuid", "rbind"], "Propagation": "rprivate", "RW": true, "Source": "/var/lib/containers/storage/volumes/fd4e6a9f601231af729313e3c82a9efaee3fa0d375634cfdbc46883798e3110b/_data", "Type": "volume"}, {"Destination": "/opt/graphite/conf", "Driver": "", "Mode": "", "Name": "", "Options": ["rbind"], "Propagation": "rprivate", "RW": true, "Source": "/opt/graphite/conf", "Type": "bind"}], "Name": "graphite", "Namespace": "", "NetworkSettings": {"Bridge": "", "EndpointID": "", "Gateway": "10.88.0.1", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "HairpinMode": false, "IPAddress": "10.88.0.191", "IPPrefixLen": 16, "IPv6Gateway": "", "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "MacAddress": "d6:bf:94:bb:31:f8", "Ports": [{"containerPort": 80, "hostIP": "", "hostPort": 80, "protocol": "tcp"}, {"containerPort": 2003, "hostIP": "", "hostPort": 2003, "protocol": "tcp"}, {"containerPort": 2004, "hostIP": "", "hostPort": 2004, "protocol": "tcp"}], "SandboxID": "", "SandboxKey": "/var/run/netns/cni-2c999ba1-e517-473f-a9e3-b2c3cf52ae87"}, "OCIConfigPath": "/var/lib/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/config.json", "OCIRuntime": "runc", "Path": "/entrypoint", "Pod": "", "ProcessLabel": "", "ResolvConfPath": "/var/run/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata/resolv.conf", "RestartCount": 0, "Rootfs": "", "State": {"ConmonPid": 12840, "Dead": false, "Error": "", "ExitCode": 0, "FinishedAt": "0001-01-01T00:00:00Z", "Healthcheck": {"FailingStreak": 0, "Log": null, "Status": ""}, "OOMKilled": false, "OciVersion": "1.0.1-dev", "Paused": false, "Pid": 12879, "Restarting": false, "Running": true, "StartedAt": "2020-06-02T12:28:28.784235249Z", "Status": "running"}, "StaticDir": "/var/lib/containers/storage/overlay-containers/530686241e65b916af9505f256772b7d88a0c51cc2337dc2a7d0c70d616d6119/userdata"}, "podman_actions": ["podman rm -f graphite", "podman run --name graphite --env REDIS_TAGDB=1 --publish 80:80 --publish 2003-2004:2003-2004 --volume /opt/graphite/conf:/opt/graphite/conf --volume graphite-data:/opt/graphite/storage --detach=True graphiteapp/graphite-statsd:1.1.7-2"], "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

(In the meantime I switched from mounting the storage folder on the local filesystem to using a volume I created via podman volume create --opt device={{ data_device }} graphite-data. It does not seem to like this either. However, this might be considered a separate issue)

The following is the new code:

- name: Install Graphite container
  containers.podman.podman_container:
    env:
      REDIS_TAGDB: 1
    image: "graphiteapp/graphite-statsd:{{ service_versions.dict.graphite.version }}"
    name: graphite
    ports:
      - "80:80"
      - "2003-2004:2003-2004"
    state: present
    volumes:
      - "/opt/graphite/conf:/opt/graphite/conf"
      - "graphite-data:/opt/graphite/storage"
sshnaidm commented 4 years ago

@jaudriga thanks, you provide very good cases :) Will look asap.

sshnaidm commented 4 years ago

@jaudriga let me understand please the image issue better, because it's not reproducible for me:

-image - graphite-statsd
+image - graphite-statsd:1.1.7-2

What is image tag in first run and what is it in second one? And what is value of {{ service_versions.dict.graphite.version }}?

jaudriga commented 4 years ago

it's not reproducible for me

@sshnaidm The issue seems to be resolved for me as well. Yeah :) . However, I am going to answer your questions in case anyone else will have a similar issue.

What is image tag in first run and what is it in second one?

The image tag was the same in the first and the second run: 1.1.7-2. I remember that, before the issue was fixed, when I ran podman inspect Image or ImageName did not mention the tag 1.1.7-2 as it does now:

# podman inspect graphite | grep graphite-statsd
        "ImageName": "docker.io/graphiteapp/graphite-statsd:1.1.7-2",
            "Image": "docker.io/graphiteapp/graphite-statsd:1.1.7-2",
                "graphiteapp/graphite-statsd:1.1.7-2"

Sadly I did not document the output of podman inspect at the time.

And what is value of {{ service_versions.dict.graphite.version }}?

Arg. Sry I forgot to replace the above service_versions variable. Its value is 1.1.7-2.

sshnaidm commented 4 years ago

@jaudriga cool, I hope last patch solved it for you. Let me know please if there are any other issues. Thanks!

jaudriga commented 4 years ago

@sshnaidm I just tried out version 1.0.3 of this collection. It is working fine! Thank you :) .

I noticed something of a follow-up issue for which I currently do not want to create a separate issue.. just wanted to let you know:

When using

- name: Update Graylog Container
   containers.podman.podman_container:
     volumes:
       - "graylog-journal:/usr/share/graylog/data/journal"
       - "graylog-plugin:/usr/share/graylog/plugin/"

The diff showed the following reproducibly:

$ ansible-playbook --vault-password-file password.txt full.yml -CD
...
TASK [graylog : Update Graylog Container] ************************************************************************************************************************************************************************************************************************************************************************************************************
--- before
+++ after
@@ -1 +1 @@
-volume - []
+volume - ['graylog-plugin:/usr/share/graylog/plugin/']

changed: [logging.exp]

However, removing the trailing / fixed this.

sshnaidm commented 4 years ago

@jaudriga yes, podman remove all slashes in inspect. So we need to strip all ending slashes in input too. Will send a fix.