Unable to start container with block device mounted on rootfs

[dann1]

Required information

• Distribution: Ubuntu
• Distribution version: 2004
• The output of "lxc info":

  config: {}
  api_extensions:
  - storage_zfs_remove_snapshots
  - container_host_shutdown_timeout
  - container_stop_priority
  - container_syscall_filtering
  - auth_pki
  - container_last_used_at
  - etag
  - patch
  - usb_devices
  - https_allowed_credentials
  - image_compression_algorithm
  - directory_manipulation
  - container_cpu_time
  - storage_zfs_use_refquota
  - storage_lvm_mount_options
  - network
  - profile_usedby
  - container_push
  - container_exec_recording
  - certificate_update
  - container_exec_signal_handling
  - gpu_devices
  - container_image_properties
  - migration_progress
  - id_map
  - network_firewall_filtering
  - network_routes
  - storage
  - file_delete
  - file_append
  - network_dhcp_expiry
  - storage_lvm_vg_rename
  - storage_lvm_thinpool_rename
  - network_vlan
  - image_create_aliases
  - container_stateless_copy
  - container_only_migration
  - storage_zfs_clone_copy
  - unix_device_rename
  - storage_lvm_use_thinpool
  - storage_rsync_bwlimit
  - network_vxlan_interface
  - storage_btrfs_mount_options
  - entity_description
  - image_force_refresh
  - storage_lvm_lv_resizing
  - id_map_base
  - file_symlinks
  - container_push_target
  - network_vlan_physical
  - storage_images_delete
  - container_edit_metadata
  - container_snapshot_stateful_migration
  - storage_driver_ceph
  - storage_ceph_user_name
  - resource_limits
  - storage_volatile_initial_source
  - storage_ceph_force_osd_reuse
  - storage_block_filesystem_btrfs
  - resources
  - kernel_limits
  - storage_api_volume_rename
  - macaroon_authentication
  - network_sriov
  - console
  - restrict_devlxd
  - migration_pre_copy
  - infiniband
  - maas_network
  - devlxd_events
  - proxy
  - network_dhcp_gateway
  - file_get_symlink
  - network_leases
  - unix_device_hotplug
  - storage_api_local_volume_handling
  - operation_description
  - clustering
  - event_lifecycle
  - storage_api_remote_volume_handling
  - nvidia_runtime
  - container_mount_propagation
  - container_backup
  - devlxd_images
  - container_local_cross_pool_handling
  - proxy_unix
  - proxy_udp
  - clustering_join
  - proxy_tcp_udp_multi_port_handling
  - network_state
  - proxy_unix_dac_properties
  - container_protection_delete
  - unix_priv_drop
  - pprof_http
  - proxy_haproxy_protocol
  - network_hwaddr
  - proxy_nat
  - network_nat_order
  - container_full
  - candid_authentication
  - backup_compression
  - candid_config
  - nvidia_runtime_config
  - storage_api_volume_snapshots
  - storage_unmapped
  - projects
  - candid_config_key
  - network_vxlan_ttl
  - container_incremental_copy
  - usb_optional_vendorid
  - snapshot_scheduling
  - container_copy_project
  - clustering_server_address
  - clustering_image_replication
  - container_protection_shift
  - snapshot_expiry
  - container_backup_override_pool
  - snapshot_expiry_creation
  - network_leases_location
  - resources_cpu_socket
  - resources_gpu
  - resources_numa
  - kernel_features
  - id_map_current
  - event_location
  - storage_api_remote_volume_snapshots
  - network_nat_address
  - container_nic_routes
  - rbac
  - cluster_internal_copy
  - seccomp_notify
  - lxc_features
  - container_nic_ipvlan
  - network_vlan_sriov
  - storage_cephfs
  - container_nic_ipfilter
  - resources_v2
  - container_exec_user_group_cwd
  - container_syscall_intercept
  - container_disk_shift
  - storage_shifted
  - resources_infiniband
  - daemon_storage
  - instances
  - image_types
  - resources_disk_sata
  - clustering_roles
  - images_expiry
  - resources_network_firmware
  - backup_compression_algorithm
  - ceph_data_pool_name
  - container_syscall_intercept_mount
  - compression_squashfs
  - container_raw_mount
  - container_nic_routed
  - container_syscall_intercept_mount_fuse
  - container_disk_ceph
  - virtual-machines
  - image_profiles
  - clustering_architecture
  - resources_disk_id
  - storage_lvm_stripes
  - vm_boot_priority
  - unix_hotplug_devices
  - api_filtering
  - instance_nic_network
  - clustering_sizing
  - firewall_driver
  - projects_limits
  - container_syscall_intercept_hugetlbfs
  - limits_hugepages
  - container_nic_routed_gateway
  - projects_restrictions
  - custom_volume_snapshot_expiry
  - volume_snapshot_scheduling
  - trust_ca_certificates
  - snapshot_disk_usage
  - clustering_edit_roles
  - container_nic_routed_host_address
  - container_nic_ipvlan_gateway
  - resources_usb_pci
  - resources_cpu_threads_numa
  - resources_cpu_core_die
  - api_os
  - resources_system
  api_status: stable
  api_version: "1.0"
  auth: trusted
  public: false
  auth_methods:
  - tls
  environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: |
  -----BEGIN CERTIFICATE-----
  -----END CERTIFICATE-----
  certificate_fingerprint: 9e617f988dfccdbe5c617f631d6f1a94c0415ed25583989f25c2d65e9c549c3c
  driver: lxc
  driver_version: 4.0.2
  firewall: nftables
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
  netnsid_getifaddrs: "false"
  seccomp_listener: "true"
  seccomp_listener_continue: "true"
  shiftfs: "false"
  uevent_injection: "true"
  unpriv_fscaps: "true"
  kernel_version: 5.4.0-21-generic
  lxc_features:
  cgroup2: "true"
  mount_injection_file: "true"
  network_gateway_device_route: "true"
  network_ipvlan: "true"
  network_l2proxy: "true"
  network_phys_macvlan_mtu: "true"
  network_veth_router: "true"
  pidfd: "true"
  seccomp_notify: "true"
  os_name: Ubuntu
  os_version: "20.04"
  project: default
  server: lxd
  server_clustered: false
  server_name: ubuntu2004-lxd-ssh-5-11-90-adfd7-2.test
  server_pid: 3484
  server_version: 4.0.1
  storage: dir
  storage_version: "1"

Issue description

Containers rootfs are located under /var/snap/lxd/common/lxd/containers/<container>/rootfs, if one would like to have a block device mounted on the rootfs, for example, when creating empty containers, the container fails to start. In lxd3 this didn't happen.

Steps to reproduce

A regular container will be created and started, the one with the block device will fail. The block device contains a copy of the rootfs of the 1st container

1. Create a normal container and dump it into a block device

   
   root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# lxc init images:alpine/edge rootfs
   Creating rootfs

The instance you are starting doesn't have any network attached to it. To create a new network, use: lxc network create To attach a network to an instance, use: lxc network attach

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# truncate -s 1000M alpine.img root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# mkfs.ext4 alpine.img mke2fs 1.45.5 (07-Jan-2020) Discarding device blocks: done
Creating filesystem with 256000 4k blocks and 64000 inodes Filesystem UUID: 0177efb6-e818-41ae-b111-667936d22a15 Superblock backups stored on blocks: 32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# mount alpine.img /mnt/ root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# cp -rpa rootfs/rootfs/* /mnt/ root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# umount /mnt

 2. Create the blocked backed container

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# lxc init images:alpine/edge alpine-block Creating alpine-block

The instance you are starting doesn't have any network attached to it. To create a new network, use: lxc network create To attach a network to an instance, use: lxc network attach

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# rm -r alpine-block/rootfs/* root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# mount alpine.img alpine-block/rootfs/

 3. Start the containers

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# lxc start rootfs root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# lxc start alpine-block Error: Failed to run: /snap/lxd/current/bin/lxd forkstart alpine-block /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/alpine-block/lxc.conf: Try lxc info --show-log alpine-block for more info

The regular one, `rootfs` starts normally, the blocked backed one, `alpine-block` fails.

# Information to attach
 - [ ] Container log (`lxc info NAME --show-log`)
The container log complains about missing `/sbin/init` but it does exist

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# ls -lh alpine-block/rootfs/sbin/init lrwxrwxrwx 1 root root 12 Jun 26 2018 alpine-block/rootfs/sbin/init -> /bin/busybox root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# ls -lh alpine-block/rootfs/bin/busybox -rwxr-xr-x 1 root root 822K Jun 5 04:45 alpine-block/rootfs/bin/busybox

root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# lxc info --show-log alpine-block Name: alpine-block Location: none Remote: unix:// Architecture: x86_64 Created: 2020/06/09 14:23 UTC Status: Stopped Type: container Profiles: default

Log:

lxc alpine-block 20200609143303.558 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.alpine-block" lxc alpine-block 20200609143303.568 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.alpine-block" lxc alpine-block 20200609143303.582 WARN cgfsng - cgroups/cgfsng.c:fchowmodat:1572 - No such file or directory - Failed to fchownat(17, memory.oom.group, 65536, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW ) lxc alpine-block 20200609143303.697 ERROR start - start.c:start:2042 - No such file or directory - Failed to exec "/sbin/init" lxc alpine-block 20200609143303.697 ERROR sync - sync.c:sync_wait:41 - An error occurred in another process (expected sequence number 7) lxc alpine-block 20200609143303.697 ERROR lxccontainer - lxccontainer.c:wait_on_daemonized_start:850 - Received container state "ABORTING" instead of "RUNNING" lxc alpine-block 20200609143303.699 ERROR start - start.c:lxc_start:1950 - Failed to spawn container "alpine-block" lxc alpine-block 20200609143303.699 WARN start - start.c:lxc_abort:1018 - No such process - Failed to send SIGKILL via pidfd 30 for process 11649 lxc 20200609143303.780 WARN commands - commands.c:lxc_cmd_rsp_recv:124 - Connection reset by peer - Failed to receive response for command "get_state"

 - [ ] Container configuration (`lxc config show NAME --expanded`)

architecture: x86_64 config: image.architecture: amd64 image.description: Alpine edge amd64 (20200608_13:00) image.os: Alpine image.release: edge image.serial: "20200608_13:00" image.type: squashfs limits.cpu: "1" limits.cpu.allowance: 50% limits.memory: 512MB security.idmap.base: "100000" security.idmap.isolated: "true" security.idmap.size: "65536" volatile.base_image: 25d4e63055e41a70ca129049e5eecfd05e1680e1eb83761bc79ff2fa0dbb0ec7 volatile.idmap.base: "100000" volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]' volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]' volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]' volatile.last_state.power: STOPPED devices: root: path: / pool: default type: disk ephemeral: false profiles:

• default stateful: false description: ""

  - [ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log)

  root@ubuntu2004-lxd-ssh-5-11-90-adfd7-2:/var/snap/lxd/common/lxd/containers# cat /var/snap/lxd/common/lxd/logs/lxd.log t=2020-06-09T13:59:23+0000 lvl=info msg="LXD 4.0.1 is starting in normal mode" path=/var/snap/lxd/common/lxd t=2020-06-09T13:59:23+0000 lvl=info msg="Kernel uid/gid map:" t=2020-06-09T13:59:23+0000 lvl=info msg=" - u 0 0 4294967295" t=2020-06-09T13:59:23+0000 lvl=info msg=" - g 0 0 4294967295" t=2020-06-09T13:59:23+0000 lvl=info msg="Configured LXD uid/gid map:" t=2020-06-09T13:59:23+0000 lvl=info msg=" - u 0 1000000 1000000000" t=2020-06-09T13:59:23+0000 lvl=info msg=" - g 0 1000000 1000000000" t=2020-06-09T13:59:23+0000 lvl=info msg="Kernel features:" t=2020-06-09T13:59:23+0000 lvl=info msg=" - netnsid-based network retrieval: yes" t=2020-06-09T13:59:23+0000 lvl=info msg=" - pidfds: no" t=2020-06-09T13:59:23+0000 lvl=info msg=" - uevent injection: yes" t=2020-06-09T13:59:23+0000 lvl=info msg=" - seccomp listener: yes" t=2020-06-09T13:59:23+0000 lvl=info msg=" - seccomp listener continue syscalls: yes" t=2020-06-09T13:59:23+0000 lvl=info msg=" - unprivileged file capabilities: yes" t=2020-06-09T13:59:23+0000 lvl=info msg=" - cgroup layout: hybrid" t=2020-06-09T13:59:23+0000 lvl=warn msg=" - Couldn't find the CGroup blkio.weight, I/O weight limits will be ignored" t=2020-06-09T13:59:23+0000 lvl=warn msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored" t=2020-06-09T13:59:23+0000 lvl=info msg=" - shiftfs support: disabled" t=2020-06-09T13:59:23+0000 lvl=info msg="Initializing local database" t=2020-06-09T13:59:23+0000 lvl=info msg="Starting /dev/lxd handler:" t=2020-06-09T13:59:23+0000 lvl=info msg=" - binding devlxd socket" socket=/var/snap/lxd/common/lxd/devlxd/sock t=2020-06-09T13:59:23+0000 lvl=info msg="REST API daemon:" t=2020-06-09T13:59:23+0000 lvl=info msg=" - binding Unix socket" inherited=true socket=/var/snap/lxd/common/lxd/unix.socket t=2020-06-09T13:59:23+0000 lvl=info msg="Initializing global database" t=2020-06-09T13:59:24+0000 lvl=info msg="Firewall loaded driver \"nftables\"" t=2020-06-09T13:59:24+0000 lvl=info msg="Initializing storage pools" t=2020-06-09T13:59:26+0000 lvl=info msg="Initializing daemon storage mounts" t=2020-06-09T13:59:26+0000 lvl=info msg="Initializing networks" t=2020-06-09T13:59:26+0000 lvl=info msg="Pruning leftover image files" t=2020-06-09T13:59:26+0000 lvl=info msg="Done pruning leftover image files" t=2020-06-09T13:59:26+0000 lvl=info msg="Loading daemon configuration" t=2020-06-09T13:59:27+0000 lvl=info msg="Started seccomp handler" path=/var/snap/lxd/common/lxd/seccomp.socket t=2020-06-09T13:59:27+0000 lvl=info msg="Pruning expired images" t=2020-06-09T13:59:27+0000 lvl=info msg="Done pruning expired images" t=2020-06-09T13:59:27+0000 lvl=info msg="Pruning expired instance backups" t=2020-06-09T13:59:27+0000 lvl=info msg="Done pruning expired instance backups" t=2020-06-09T13:59:27+0000 lvl=info msg="Expiring log files" t=2020-06-09T13:59:27+0000 lvl=info msg="Done expiring log files" t=2020-06-09T13:59:27+0000 lvl=info msg="Updating images" t=2020-06-09T13:59:27+0000 lvl=info msg="Done updating images" t=2020-06-09T13:59:27+0000 lvl=info msg="Updating instance types" t=2020-06-09T13:59:27+0000 lvl=info msg="Done updating instance types" t=2020-06-09T14:00:20+0000 lvl=info msg="Creating container" ephemeral=false name=alpine2 project=default t=2020-06-09T14:00:20+0000 lvl=info msg="Created container" ephemeral=false name=alpine2 project=default t=2020-06-09T14:00:30+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:00:31+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:01:05+0000 lvl=eror msg="Failed to retrieve network information via netlink" container=alpine2 pid=3749 t=2020-06-09T14:01:47+0000 lvl=eror msg="Running apparmor" action=r err="Failed to run: apparmor_parser -rWL /var/snap/lxd/common/lxd/security/apparmor/cache /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2: AppArmor parser error for /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 in /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 at line 507: syntax error, unexpected TOK_CLOSE, expecting TOK_MODE" output= t=2020-06-09T14:01:55+0000 lvl=eror msg="Running apparmor" action=Q err="Failed to run: apparmor_parser -QWL /var/snap/lxd/common/lxd/security/apparmor/cache /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2: AppArmor parser error for /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 in /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 at line 507: syntax error, unexpected TOK_CLOSE, expecting TOK_MODE" output= t=2020-06-09T14:02:21+0000 lvl=eror msg="Running apparmor" action=Q err="Failed to run: apparmor_parser -QWL /var/snap/lxd/common/lxd/security/apparmor/cache /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2: AppArmor parser error for /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 in /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 at line 507: syntax error, unexpected TOK_CLOSE, expecting TOK_MODE" output= t=2020-06-09T14:02:55+0000 lvl=info msg="Shutting down container" action=shutdown created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default timeout=-1s used=2020-06-09T14:00:31+0000 t=2020-06-09T14:02:58+0000 lvl=eror msg="Running apparmor" action=R err="Failed to run: apparmor_parser -RWL /var/snap/lxd/common/lxd/security/apparmor/cache /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2: AppArmor parser error for /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 in /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 at line 507: syntax error, unexpected TOK_CLOSE, expecting TOK_MODE" output= t=2020-06-09T14:02:58+0000 lvl=eror msg="Failed to destroy apparmor namespace" container=alpine2 err="Failed to run: apparmor_parser -RWL /var/snap/lxd/common/lxd/security/apparmor/cache /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2: AppArmor parser error for /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 in /var/snap/lxd/common/lxd/security/apparmor/profiles/lxd-alpine2 at line 507: syntax error, unexpected TOK_CLOSE, expecting TOK_MODE" t=2020-06-09T14:02:58+0000 lvl=info msg="Shut down container" action=shutdown created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default timeout=-1s used=2020-06-09T14:00:31+0000 t=2020-06-09T14:03:38+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:00:31+0000 t=2020-06-09T14:03:38+0000 lvl=eror msg="Failed starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:00:31+0000 t=2020-06-09T14:03:39+0000 lvl=info msg="Container initiated stop" action=stop created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:03:38+0000 t=2020-06-09T14:03:39+0000 lvl=info msg="Shut down container" action=stop created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:03:38+0000 t=2020-06-09T14:04:14+0000 lvl=info msg="Creating container" ephemeral=false name=alpine3 project=default t=2020-06-09T14:04:14+0000 lvl=info msg="Created container" ephemeral=false name=alpine3 project=default t=2020-06-09T14:04:47+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:03:38+0000 t=2020-06-09T14:04:47+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:03:38+0000 t=2020-06-09T14:04:50+0000 lvl=info msg="Shutting down container" action=shutdown created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default timeout=-1s used=2020-06-09T14:04:47+0000 t=2020-06-09T14:04:53+0000 lvl=info msg="Shut down container" action=shutdown created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default timeout=-1s used=2020-06-09T14:04:47+0000 t=2020-06-09T14:05:04+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:04:47+0000 t=2020-06-09T14:05:05+0000 lvl=eror msg="Failed starting container" action=start created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:04:47+0000 t=2020-06-09T14:05:05+0000 lvl=info msg="Container initiated stop" action=stop created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:05:05+0000 t=2020-06-09T14:05:05+0000 lvl=info msg="Shut down container" action=stop created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default stateful=false used=2020-06-09T14:05:05+0000 t=2020-06-09T14:05:57+0000 lvl=info msg="Creating container" ephemeral=false name=rootfs project=default t=2020-06-09T14:05:57+0000 lvl=info msg="Created container" ephemeral=false name=rootfs project=default t=2020-06-09T14:06:08+0000 lvl=info msg="Deleting container" created=2020-06-09T13:53:14+0000 ephemeral=false name=alpine project=default used=2020-06-09T13:54:24+0000 t=2020-06-09T14:06:08+0000 lvl=info msg="Deleted container" created=2020-06-09T13:53:14+0000 ephemeral=false name=alpine project=default used=2020-06-09T13:54:24+0000 t=2020-06-09T14:06:08+0000 lvl=info msg="Deleting container" created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default used=2020-06-09T14:05:05+0000 t=2020-06-09T14:06:08+0000 lvl=info msg="Deleted container" created=2020-06-09T14:00:20+0000 ephemeral=false name=alpine2 project=default used=2020-06-09T14:05:05+0000 t=2020-06-09T14:06:08+0000 lvl=info msg="Deleting container" created=2020-06-09T14:04:14+0000 ephemeral=false name=alpine3 project=default used=1970-01-01T00:00:00+0000 t=2020-06-09T14:06:08+0000 lvl=info msg="Deleted container" created=2020-06-09T14:04:14+0000 ephemeral=false name=alpine3 project=default used=1970-01-01T00:00:00+0000 t=2020-06-09T14:06:17+0000 lvl=info msg="Creating container" ephemeral=false name=alpine project=default t=2020-06-09T14:06:17+0000 lvl=info msg="Created container" ephemeral=false name=alpine project=default t=2020-06-09T14:06:24+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:06:24+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:06:47+0000 lvl=info msg="Shutting down container" action=shutdown created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default timeout=-1s used=2020-06-09T14:06:24+0000 t=2020-06-09T14:06:50+0000 lvl=info msg="Shut down container" action=shutdown created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default timeout=-1s used=2020-06-09T14:06:24+0000 t=2020-06-09T14:06:55+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:06:24+0000 t=2020-06-09T14:06:55+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:06:24+0000 t=2020-06-09T14:06:59+0000 lvl=info msg="Shutting down container" action=shutdown created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default timeout=-1s used=2020-06-09T14:06:55+0000 t=2020-06-09T14:07:02+0000 lvl=info msg="Shut down container" action=shutdown created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default timeout=-1s used=2020-06-09T14:06:55+0000 t=2020-06-09T14:07:36+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:06:55+0000 t=2020-06-09T14:07:37+0000 lvl=eror msg="Failed starting container" action=start created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:06:55+0000 t=2020-06-09T14:07:37+0000 lvl=info msg="Container initiated stop" action=stop created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:07:36+0000 t=2020-06-09T14:07:37+0000 lvl=info msg="Shut down container" action=stop created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default stateful=false used=2020-06-09T14:07:36+0000 t=2020-06-09T14:19:49+0000 lvl=info msg="Deleting container" created=2020-06-09T12:52:51+0000 ephemeral=false name=one-8 project=default used=2020-06-09T12:55:15+0000 t=2020-06-09T14:19:49+0000 lvl=info msg="Deleted container" created=2020-06-09T12:52:51+0000 ephemeral=false name=one-8 project=default used=2020-06-09T12:55:15+0000 t=2020-06-09T14:19:53+0000 lvl=info msg="Deleting container" created=2020-06-09T14:05:57+0000 ephemeral=false name=rootfs project=default used=1970-01-01T00:00:00+0000 t=2020-06-09T14:19:53+0000 lvl=info msg="Deleted container" created=2020-06-09T14:05:57+0000 ephemeral=false name=rootfs project=default used=1970-01-01T00:00:00+0000 t=2020-06-09T14:19:55+0000 lvl=info msg="Deleting container" created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default used=2020-06-09T14:07:36+0000 t=2020-06-09T14:19:55+0000 lvl=info msg="Deleted container" created=2020-06-09T14:06:17+0000 ephemeral=false name=alpine project=default used=2020-06-09T14:07:36+0000 t=2020-06-09T14:20:03+0000 lvl=info msg="Creating container" ephemeral=false name=rootfs project=default t=2020-06-09T14:20:03+0000 lvl=info msg="Created container" ephemeral=false name=rootfs project=default t=2020-06-09T14:20:06+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:20:07+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:20:25+0000 lvl=info msg="Stopping container" action=stop created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default stateful=false used=2020-06-09T14:20:07+0000 t=2020-06-09T14:20:26+0000 lvl=info msg="Stopped container" action=stop created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default stateful=false used=2020-06-09T14:20:07+0000 t=2020-06-09T14:20:26+0000 lvl=info msg="Deleting container" created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default used=2020-06-09T14:20:07+0000 t=2020-06-09T14:20:26+0000 lvl=info msg="Deleted container" created=2020-06-09T14:20:03+0000 ephemeral=false name=rootfs project=default used=2020-06-09T14:20:07+0000 t=2020-06-09T14:21:18+0000 lvl=info msg="Creating container" ephemeral=false name=rootfs project=default t=2020-06-09T14:21:18+0000 lvl=info msg="Created container" ephemeral=false name=rootfs project=default t=2020-06-09T14:23:47+0000 lvl=info msg="Creating container" ephemeral=false name=alpine-block project=default t=2020-06-09T14:23:47+0000 lvl=info msg="Created container" ephemeral=false name=alpine-block project=default t=2020-06-09T14:24:20+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:21:18+0000 ephemeral=false name=rootfs project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:24:21+0000 lvl=info msg="Started container" action=start created=2020-06-09T14:21:18+0000 ephemeral=false name=rootfs project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:24:25+0000 lvl=info msg="Starting container" action=start created=2020-06-09T14:23:47+0000 ephemeral=false name=alpine-block project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:24:25+0000 lvl=eror msg="Failed starting container" action=start created=2020-06-09T14:23:47+0000 ephemeral=false name=alpine-block project=default stateful=false used=1970-01-01T00:00:00+0000 t=2020-06-09T14:24:26+0000 lvl=info msg="Container initiated stop" action=stop created=2020-06-09T14:23:47+0000 ephemeral=false name=alpine-block project=default stateful=false used=2020-06-09T14:24:25+0000 t=2020-06-09T14:24:26+0000 lvl=info msg="Shut down container" action=stop created=2020-06-09T14:23:47+0000 ephemeral=false name=alpine-block project=default stateful=false used=2020-06-09T14:24:25+0000

[stgraber]

This is a very very unsupported setup. This type of hack will completely break any of the uid/gid mapping logic, snapshots, file transfers and likely a variety of other features I'm just not thinking of right now.

The reason for what you're seeing is because of the snap's mount namespace. Basically your mount call isn't visible inside the namespace so it just sees an empty directory. This can be worked around if you do the mount inside the right namespace, but again, not something we'll support.