search

containers / podman-compose

a script to run docker-compose.yml using podman
GNU General Public License v2.0
5.09k stars 483 forks source link

rootless troubles with bind mounts #124

Open Pusikas opened 4 years ago

Pusikas commented 4 years ago

This is my docker-compose.yml:

version: "3"

services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: mongo:3
    volumes:
      - mongo_data:/data/db

  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.x/docker.html
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.5
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

# Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/
volumes:
  mongo_data:
  es_data:

This is mostly from some example for setting up Graylog found on the web. We are trying to run rootless on RHEL 8.1. System uses cgroup v1. This is the uid map:

$ podman unshare cat /proc/self/uid_map
         0       1001          1
         1     165536      65536

This is out podman:

$ podman version
Version:            1.6.4
RemoteAPI Version:  1
Go Version:         go1.12.12
OS/Arch:            linux/amd64

$ podman info
host:
  BuildahVersion: 1.12.0-dev
  CgroupVersion: v1
  Conmon:
    package: conmon-2.0.6-1.module+el8.1.1+5259+bcdd613a.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.6, commit: 6ffbb2ec70dbe5ba56e4bfde946fb04f19dd8bbf'
  Distribution:
    distribution: '"rhel"'
    version: "8.1"
  IDMappings:
    gidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
  MemFree: 9384361984
  MemTotal: 16645152768
  OCIRuntime:
    name: runc
    package: runc-1.0.0-64.rc9.module+el8.1.1+5259+bcdd613a.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree: 8000880640
  SwapTotal: 8002727936
  arch: amd64
  cpus: 16
  eventlogger: journald
  hostname: dlmbrpdm001.cslg1.cslg.net
  kernel: 4.18.0-147.el8.x86_64
  os: linux
  rootless: true
  slirp4netns:
    Executable: /usr/bin/slirp4netns
    Package: slirp4netns-0.4.2-2.git21fdece.module+el8.1.1+5460+3ac089c3.x86_64
    Version: |-
      slirp4netns version 0.4.2+dev
      commit: 21fdece2737dc24ffa3f01a341b8a6854f8b13b4
  uptime: 194h 43m 12.78s (Approximately 8.08 days)
registries:
  blocked: null
  insecure: null
  search:
  - registry.access.redhat.com
  - registry.fedoraproject.org
  - registry.centos.org
  - docker.io
store:
  ConfigFile: /home/plessy/.config/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-0.7.2-1.module+el8.1.1+5259+bcdd613a.x86_64
      Version: |-
        fuse-overlayfs: version 0.7.2
        FUSE library version 3.2.1
        using FUSE kernel interface version 7.26
  GraphRoot: /home/plessy/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 4
  RunRoot: /tmp/run-1001
  VolumePath: /home/plessy/.local/share/containers/storage/volumes

Mongo seems to use uid 999 inside the container, elasticsearch uses 1000.

We are using podman-compose 0.1.6dev.

When I start do a podman-compose up, then MongoDB starts up nicely. Elasticsearch gives me the following error: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes.

This is the dry run:

$ podman-compose --dry-run up
podman pod create --name=frank --share net
podman volume inspect frank_mongo_data || podman volume create frank_mongo_data
podman create --name=frank_mongodb_1 --pod=frank --label io.podman.compose.config-hash=123 --label io.podman.compose.project=frank --label io.podman.compose.version=0.0.1 --label com.docker.compose.container-number=1 --label com.docker.compose.service=mongodb --mount type=bind,source=/home/plessy/.local/share/containers/storage/volumes/frank_mongo_data/_data,destination=/data/db,bind-propagation=z --add-host mongodb:127.0.0.1 --add-host frank_mongodb_1:127.0.0.1 --add-host elasticsearch:127.0.0.1 --add-host frank_elasticsearch_1:127.0.0.1 mongo:3
podman volume inspect frank_es_data || podman volume create frank_es_data
podman create --name=frank_elasticsearch_1 --pod=frank --label io.podman.compose.config-hash=123 --label io.podman.compose.project=frank --label io.podman.compose.version=0.0.1 --label com.docker.compose.container-number=1 --label com.docker.compose.service=elasticsearch -e http.host=0.0.0.0 -e transport.host=localhost -e network.host=0.0.0.0 -e ES_JAVA_OPTS=-Xms512m -Xmx512m --mount type=bind,source=/home/plessy/.local/share/containers/storage/volumes/frank_es_data/_data,destination=/usr/share/elasticsearch/data,bind-propagation=z --add-host mongodb:127.0.0.1 --add-host frank_mongodb_1:127.0.0.1 --add-host elasticsearch:127.0.0.1 --add-host frank_elasticsearch_1:127.0.0.1 docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.5

I do not really understand why podman-compose first creates volumes and then bind mounts their directories. This seems to make a difference here.

$ ls -alF $HOME/.local/share/containers/storage/volumes/frank_mongo_data
total 8
drwx------  3 plessy plessy   19 Mar  6 11:53 ./
drwx------ 18 plessy plessy 4096 Mar  6 11:53 ../
drwxr-xr-x  4 166534 plessy 4096 Mar  6 11:53 _data/

$ ls -alF $HOME/.local/share/containers/storage/volumes/frank_es_data
total 4
drwx------  3 plessy plessy   19 Mar  6 11:53 ./
drwx------ 18 plessy plessy 4096 Mar  6 11:53 ../
drwxr-xr-x  2 plessy plessy    6 Mar  6 11:53 _data/

As you can see, the _data dir's user was correctly changed for the MongoDB volume, but not for the Elasticsearch volume. If I stop the elasticsearch container, run podman unshare chown 1000:1000 /home/plessy/.local/share/containers/storage/volumes/frank_es_data/_data, and then then start it up again, then it works. It also works when I change the --mount type=bind,source=/home/plessy/.local/share/containers/storage/volumes/frank_es_data/_data,destination=/usr/share/elasticsearch/data,bind-propagation=z to --mount type=volume,source=frank_es_data,destination=/usr/share/elasticsearch/data.

So, my question is: What am I doing wrong? And why doesn't podman-compose just use the containers it creates? Instead of using bind mounts on them?

p1r4t3-s4il0r commented 1 year ago

Hello.

I'm having the same issue as you : java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes

Because my data folder is empty, on both my host and container. Do you have any idea why ?

I'm not using podman, but docker rootless which should be pretty similar.

Thanks