search

containers / krunvm

Create microVMs from OCI images
Apache License 2.0
1.43k stars 42 forks source link

Unable to map ports with podman 3.4.0 #11

Closed monken closed 2 years ago

monken commented 3 years ago

Hi,

I've been exploring krunvm to run a podman service that I will use from my M1 MacBook. I've been able to get it to work, except that port mappings are being ignore. Here are the steps to reproduce it:

Any help is greatly appreciated!

host:
  arch: arm64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc34.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: container
    version: "34"
  eventLogger: file
  hostname: fedora
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.10
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 629608448
  memTotal: 1038118912
  ociRuntime:
    name: crun
    package: crun-1.0-1.fc34.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.0
      commit: 139dc6971e2f1d931af520188763e984d6cdfbf8
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc34.aarch64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 0
  swapTotal: 0
  uptime: 19m 14.23s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: <unknown>
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 3
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.0
  Built: 1633030810
  BuiltTime: Thu Sep 30 19:40:10 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/arm64
  Version: 3.4.0

cat /etc/cni/net.d/87-podman.conflist

{
  "cniVersion": "0.4.0",
  "name": "podman",
  "plugins": [
    {
      "type": "bridge",
      "bridge": "cni-podman0",
      "isGateway": true,
      "ipMasq": true,
      "hairpinMode": true,
      "ipam": {
        "type": "host-local",
        "routes": [{ "dst": "0.0.0.0/0" }],
        "ranges": [
          [
            {
              "subnet": "10.88.0.0/16",
              "gateway": "10.88.0.1"
            }
          ]
        ]
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type": "firewall"
    },
    {
      "type": "tuning"
    }
  ]
}

cat /etc/containers/containers.conf

[containers]
default_capabilities = [
    "CHOWN",
    "DAC_OVERRIDE",
    "FOWNER",
    "FSETID",
    "KILL",
    "NET_BIND_SERVICE",
    "SETFCAP",
    "SETGID",
    "SETPCAP",
    "SETUID",
    "SYS_CHROOT"
]
default_sysctls = [
 "net.ipv4.ping_group_range=0 0",
]
log_driver = "k8s-file"
netns = "host"
[secrets]
[secrets.opts]
[network]
[engine]
cgroup_manager = "cgroupfs"
events_logger = "file"
[engine.runtimes]
[engine.volume_plugins]
slp commented 3 years ago

Hi @monken,

Currently, podman's OCI networking doesn't play nice with libkrun's TSI (Transparent Socket Impersonation). The workaround is using host networking with podman. Something like this should work fine:

podman run -ti --rm --net host -p 8080:80 nginx
monken commented 3 years ago

@slp, thank you so much for your response. I really appreciate your support. I tried this before but the port mapping was ignored. I ended up being able to access the nginx server on port 80 but not on the mapped port 8080. podman ps would also not show the port mapping.

slp commented 3 years ago

The problem here is that we're dealing with double mapping, first by krunvm and then by podman. By passing -net host to podman, we inhibit the second mapping, meaning nginx will listen on port 80 inside the VM. Then, we just need to configure the port mapping in krunvm. Something like this should do the trick:

krunvm changevm LVM_NAME -p 8080:80

Then, after starting nginx with podman inside the lightweight VM, you should be able to access it from Safari on port 8080.

monken commented 3 years ago

Perfect, that works. Would be nice to have native port mapping though to support running multiple containers that expose the same port.