search

canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.32k stars 925 forks source link

Cannot apply limits.disk.priority - blkio.weight cgroup controller is missing #7420

Closed brunswyck closed 4 years ago

brunswyck commented 4 years ago

Required information

Issue description

Cannot apply limits.disk.priority When I do, the container won't start

Steps to reproduce

  1. Step one lxc profile set testcon limits.disk.priority
  2. Step two lxc start testcon
  3. Step three container won't start, unset limits.disk.priority with lxc profile unset testcon limits.disk.priority

Information to attach

Log:

lxc testcon 20200525054501.630 ERROR cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.testcon" lxc testcon 20200525054501.645 ERROR cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.testcon" lxc testcon 20200525054501.654 ERROR utils - utils.c:lxc_can_use_pidfd:1855 - Invalid argument - Kernel does not support waiting on processes through pidfds lxc testcon 20200525054501.720 WARN cgfsng - cgroups/cgfsng.c:fchowmodat:1455 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW ) lxc 20200525054502.636 WARN commands - commands.c:lxc_cmd_rsp_recv:122 - Connection reset by peer - Failed to receive response for command "get_cgroup" lxc 20200525054502.636 WARN commands - commands.c:lxc_cmd_rsp_recv:122 - Connection reset by peer - Failed to receive response for command "get_state"


 - [ ] Container configuration (`lxc config show NAME --expanded`)

$ lxc config show testcon architecture: x86_64 config: image.architecture: amd64 image.description: ubuntu 18.04 LTS amd64 (release) (20200519.1) image.label: release image.os: ubuntu image.release: bionic image.serial: "20200519.1" image.type: squashfs image.version: "18.04" user.network-config: |-

cloud-config

version: 1
config:
  - type: physical
    name: eth0
    subnets:
      - type: static
        address: 10.10.2.1/16
        gateway: 10.10.0.1
        control: manual
        dns_nameservers:
          - 10.10.0.1
          - 213.redacted
      - type: static6
        address: 2001:redacted::1:b1/128
        gateway: 2001:redacted::1:1
        control: manual
        dns_nameservers:
          - 2001:redacted::1:1
          - 2001:redacted::1

volatile.base_image: 70d3dcaabcffb1aa1644d0ce866efcb141742179e94ad72aefb8d3502338a71f volatile.eth0.hwaddr: 00:16:3e:59:6b:1e volatile.idmap.base: "0" volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]' volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]' volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]' volatile.last_state.power: STOPPED devices: {} ephemeral: false profiles:

testcon container log (var/snap/lxd/common/lxd/logs/tescon.lxc.log

lxc testcon 20200525060142.477 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.testcon"¬              
lxc testcon 20200525060142.480 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.testcon"¬              
lxc testcon 20200525060142.481 ERROR    utils - utils.c:lxc_can_use_pidfd:1855 - Invalid argument - Kernel does not support waiting on processes through pidfds¬                                  
lxc testcon 20200525060142.485 WARN     cgfsng - cgroups/cgfsng.c:fchowmodat:1455 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )¬                                                                                                                                                                                      
lxc 20200525060143.948 WARN     commands - commands.c:lxc_cmd_rsp_recv:122 - Connection reset by peer - Failed to receive response for command "get_cgroup"¬                                      
lxc 20200525060143.948 WARN     commands - commands.c:lxc_cmd_rsp_recv:122 - Connection reset by peer - Failed to receive response for command "get_state"¬ 
$ lxc --debug start testcon
DBUG[05-25|06:26:38] Connecting to a local LXD over a Unix socket 
DBUG[05-25|06:26:38] Sending request to LXD                   method=GET url=http://unix.socket/1.0 etag=
DBUG[05-25|06:26:38] Got response struct from LXD 
DBUG[05-25|06:26:38] 
    {
        "config": {},
        "api_extensions": [
            "storage_zfs_remove_snapshots",
            "container_host_shutdown_timeout",
            "container_stop_priority",
            "container_syscall_filtering",
            "auth_pki",
            "container_last_used_at",
            "etag",
            "patch",
            "usb_devices",
            "https_allowed_credentials",
            "image_compression_algorithm",
            "directory_manipulation",
            "container_cpu_time",
            "storage_zfs_use_refquota",
            "storage_lvm_mount_options",
            "network",
            "profile_usedby",
            "container_push",
            "container_exec_recording",
            "certificate_update",
            "container_exec_signal_handling",
            "gpu_devices",
            "container_image_properties",
            "migration_progress",
            "id_map",
            "network_firewall_filtering",
            "network_routes",
            "storage",
            "file_delete",
            "file_append",
            "network_dhcp_expiry",
            "storage_lvm_vg_rename",
            "storage_lvm_thinpool_rename",
            "network_vlan",
            "image_create_aliases",
            "container_stateless_copy",
            "container_only_migration",
            "storage_zfs_clone_copy",
            "unix_device_rename",
            "storage_lvm_use_thinpool",
            "storage_rsync_bwlimit",
            "network_vxlan_interface",
            "storage_btrfs_mount_options",
            "entity_description",
            "image_force_refresh",
            "storage_lvm_lv_resizing",
            "id_map_base",
            "file_symlinks",
            "container_push_target",
            "network_vlan_physical",
            "storage_images_delete",
            "container_edit_metadata",
            "container_snapshot_stateful_migration",
            "storage_driver_ceph",
            "storage_ceph_user_name",
            "resource_limits",
            "storage_volatile_initial_source",
            "storage_ceph_force_osd_reuse",
            "storage_block_filesystem_btrfs",
            "resources",
            "kernel_limits",
            "storage_api_volume_rename",
            "macaroon_authentication",
            "network_sriov",
            "console",
            "restrict_devlxd",
            "migration_pre_copy",
            "infiniband",
            "maas_network",
            "devlxd_events",
            "proxy",
            "network_dhcp_gateway",
            "file_get_symlink",
            "network_leases",
            "unix_device_hotplug",
            "storage_api_local_volume_handling",
            "operation_description",
            "clustering",
            "event_lifecycle",
            "storage_api_remote_volume_handling",
            "nvidia_runtime",
            "container_mount_propagation",
            "container_backup",
            "devlxd_images",
            "container_local_cross_pool_handling",
            "proxy_unix",
            "proxy_udp",
            "clustering_join",
            "proxy_tcp_udp_multi_port_handling",
            "network_state",
            "proxy_unix_dac_properties",
            "container_protection_delete",
            "unix_priv_drop",
            "pprof_http",
            "proxy_haproxy_protocol",
            "network_hwaddr",
            "proxy_nat",
            "network_nat_order",
            "container_full",
            "candid_authentication",
            "backup_compression",
            "candid_config",
            "nvidia_runtime_config",
            "storage_api_volume_snapshots",
            "storage_unmapped",
            "projects",
            "candid_config_key",
            "network_vxlan_ttl",
            "container_incremental_copy",
            "usb_optional_vendorid",
            "snapshot_scheduling",
            "container_copy_project",
            "clustering_server_address",
            "clustering_image_replication",
            "container_protection_shift",
            "snapshot_expiry",
            "container_backup_override_pool",
            "snapshot_expiry_creation",
            "network_leases_location",
            "resources_cpu_socket",
            "resources_gpu",
            "resources_numa",
            "kernel_features",
            "id_map_current",
            "event_location",
            "storage_api_remote_volume_snapshots",
            "network_nat_address",
            "container_nic_routes",
            "rbac",
            "cluster_internal_copy",
            "seccomp_notify",
            "lxc_features",
            "container_nic_ipvlan",
            "network_vlan_sriov",
            "storage_cephfs",
            "container_nic_ipfilter",
            "resources_v2",
            "container_exec_user_group_cwd",
            "container_syscall_intercept",
            "container_disk_shift",
            "storage_shifted",
            "resources_infiniband",
            "daemon_storage",
            "instances",
            "image_types",
            "resources_disk_sata",
            "clustering_roles",
            "images_expiry",
            "resources_network_firmware",
            "backup_compression_algorithm",
            "ceph_data_pool_name",
            "container_syscall_intercept_mount",
            "compression_squashfs",
            "container_raw_mount",
            "container_nic_routed",
            "container_syscall_intercept_mount_fuse",
            "container_disk_ceph",
            "virtual-machines",
            "image_profiles",
            "clustering_architecture",
            "resources_disk_id",
            "storage_lvm_stripes",
            "vm_boot_priority",
            "unix_hotplug_devices",
            "api_filtering",
            "instance_nic_network",
            "clustering_sizing",
            "firewall_driver",
            "projects_limits",
            "container_syscall_intercept_hugetlbfs",
            "limits_hugepages",
            "container_nic_routed_gateway",
            "projects_restrictions",
            "custom_volume_snapshot_expiry",
            "volume_snapshot_scheduling",
            "trust_ca_certificates",
            "snapshot_disk_usage",
            "clustering_edit_roles",
            "container_nic_routed_host_address",
            "container_nic_ipvlan_gateway",
            "resources_usb_pci",
            "resources_cpu_threads_numa",
            "resources_cpu_core_die",
            "api_os",
            "container_nic_routed_host_table",
            "container_nic_ipvlan_host_table",
            "container_nic_ipvlan_mode",
            "resources_system",
            "images_push_relay"
        ],
        "api_status": "stable",
        "api_version": "1.0",
        "auth": "trusted",
        "public": false,
        "auth_methods": [
            "tls"
        ],
        "environment": {
            "addresses": [],
            "architectures": [
                "x86_64",
                "i686"
            ],
            "certificate": "-----BEGIN CERTIFICATE-----\redacted\n-----END CERTIFICATE-----\n",
            "certificate_fingerprint": "33736redacted64134",
            "driver": "lxc",
            "driver_version": "4.0.2",
            "firewall": "xtables",
            "kernel": "Linux",
            "kernel_architecture": "x86_64",
            "kernel_features": {
                "netnsid_getifaddrs": "true",
                "seccomp_listener": "true",
                "seccomp_listener_continue": "true",
                "shiftfs": "false",
                "uevent_injection": "true",
                "unpriv_fscaps": "true"
            },
            "kernel_version": "5.3.0-53-generic",
            "lxc_features": {
                "cgroup2": "true",
                "mount_injection_file": "true",
                "network_gateway_device_route": "true",
                "network_ipvlan": "true",
                "network_l2proxy": "true",
                "network_phys_macvlan_mtu": "true",
                "network_veth_router": "true",
                "seccomp_notify": "true"
            },
            "os_name": "Ubuntu",
            "os_version": "18.04",
            "project": "default",
            "server": "lxd",
            "server_clustered": false,
            "server_name": "host",
            "server_pid": 1518,
            "server_version": "4.1",
            "storage": "dir",
            "storage_version": "1"
        }
    } 
DBUG[05-25|06:26:38] Sending request to LXD                   method=GET url=http://unix.socket/1.0/instances/testcon etag=
DBUG[05-25|06:26:38] Got response struct from LXD 
DBUG[05-25|06:26:38] 
    {
        "architecture": "x86_64",
        "config": {
            "image.architecture": "amd64",
            "image.description": "ubuntu 18.04 LTS amd64 (release) (20200519.1)",
            "image.label": "release",
            "image.os": "ubuntu",
            "image.release": "bionic",
            "image.serial": "20200519.1",
            "image.type": "squashfs",
            "image.version": "18.04",
            "user.network-config": "#cloud-config\nversion: 1\nconfig:\n  - type: physical\n    name: eth0\n    subnets:\n      - type: static\n        address: 10.10.2.1/16\n        gateway: 10.10.0.1\n        control: manual\n        dns_nameservers:\n          - 10.10.0.1\n          - 213.redacted\n      - type: static6\n        address: 2001:redacted::1:b1/128\n        gateway: 2001:redacted::1:1\n        control: manual\n        dns_nameservers:\n          - 2001:redacted::1:1\n          - 2001:41d0:3:163::1",
            "volatile.base_image": "70d3dcaabcffb1aa1644d0ce866efcb141742179e94ad72aefb8d3502338a71f",
            "volatile.eth0.hwaddr": "00:16:3e:59:6b:1e",
            "volatile.idmap.base": "0",
            "volatile.idmap.current": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.idmap.next": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.last_state.idmap": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.last_state.power": "STOPPED"
        },
        "devices": {},
        "ephemeral": false,
        "profiles": [
            "dev",
            "testcon"
        ],
        "stateful": false,
        "description": "",
        "created_at": "2020-05-24T19:57:00.248990143Z",
        "expanded_config": {
            "boot.autostart": "True",
            "boot.autostart.delay": "3",
            "boot.autostart.priority": "100",
            "boot.stop.priority": "99",
            "image.architecture": "amd64",
            "image.description": "ubuntu 18.04 LTS amd64 (release) (20200519.1)",
            "image.label": "release",
            "image.os": "ubuntu",
            "image.release": "bionic",
            "image.serial": "20200519.1",
            "image.type": "squashfs",
            "image.version": "18.04",
            "limits.cpu": "2",
            "limits.cpu.allowance": "10ms/100ms",
            "limits.cpu.priority": "10",
            "limits.disk.priority": "9",
            "limits.memory": "1GB",
            "limits.memory.enforce": "soft",
            "limits.network.priority": "10",
            "user.network-config": "#cloud-config\nversion: 1\nconfig:\n  - type: physical\n    name: eth0\n    subnets:\n      - type: static\n        address: 10.10.2.1/16\n        gateway: 10.10.0.1\n        control: manual\n        dns_nameservers:\n          - 10.10.0.1\n          - 213.redacted\n      - type: static6\n        address: 2001:redacted::1:b1/128\n        gateway: 2001:redacted::1:1\n        control: manual\n        dns_nameservers:\n          - 2001:redacted::1:1\n          - 2001:redacted::1",
            "user.user-data": "#cloud-config\nusers:\n  - name: dadude\n    sudo: ALL=(ALL) NOPASSWD:ALL\n    primary_group: dadude\n    groups:\n      - ['sudo']\n    ssh-authorized-keys:\n      - ssh-ed25519 redacted
dadude@dahost\n    lock_passwd: false\n    shell: /bin/bash\n    ssh_pwauth: false\n    # mkpasswd --rounds=32768 --method=SHA-512\n    passwd: redacted   - name: dadude2\n    sudo: ALL=(ALL) NOPASSWD:ALL\n    primary_group: dadude2\n    groups:\n      - ['sudo']\n    ssh-authorized-keys:\n      - ssh-ed25519 redacted dadude2@host\n    lock_passwd: false\n    shell: /bin/bash\n    ssh_pwauth: false\n    # mkpasswd --rounds=32768 --method=SHA-512\n    passwd: redacted  # run 'apt-get update'\npackage_update: true\n# Install required packages\npackages:\n  - fortune-mod\n  - cowsay\n  - fortune-anarchism\n  - vim\n  write_files:\n  - path: \"/etc/update-motd.d/99-footer\"\n    permissions: \"0755\"\n    owner: \"root\"\n    content: |\n      #!/bin/bash\n      exec /usr/games/fortune | /usr/games/cowsay -n\nruncmd:\n  - sed -i -E 's/^#?Port\\s22/Port 11111/' /etc/ssh/sshd_config\n  - sed -i -E '/^Port.*$/iProtocol 2' /etc/ssh/sshd_config\n  - sed -i -E '/#HostKey[[:space:]].*host_ed25519_key$/s/^#//g' /etc/ssh/sshd_config\n  - sed -i -E 's/#PermitRootLogin\\s.*/PermitRootLogin no/' /etc/ssh/sshd_configi\n  - sed -i -E 's/^#?LoginGraceTime\\s[[:digit:]]m$/LoginGraceTime 1m/' /etc/ssh/sshd_config\n  - sed -i -E '/StrictModes/s/^#//' /etc/ssh/sshd_config\n  - sed -i -E '/MaxAuthTries/s/^#//' /etc/ssh/sshd_config\n  - sed -i -E 's/^#?MaxSessions\\s[[:digit:]]+/MaxSessions 4/' /etc/ssh/sshd_config\n  - sed -i -E '/PubkeyAuthentication/s/^#//' /etc/ssh/sshd_config\n  - sed -i -E '/HostbasedAuthentication\\sno/s/^#//' /etc/ssh/sshd_config\n  - ufw allow 22/tcp\n  - yes | sudo ufw enable\n  - systemctl restart ssh\nfinal_message: \"The system is finally up, after $UPTIME seconds\"\n",
            "volatile.base_image": "70d3dcaabcffb1aa1644d0ce866efcb141742179e94ad72aefb8d3502338a71f",
            "volatile.eth0.hwaddr": "00:16:3e:59:6b:11",
            "volatile.idmap.base": "0",
            "volatile.idmap.current": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.idmap.next": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.last_state.idmap": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
            "volatile.last_state.power": "STOPPED"
        },
        "expanded_devices": {
            "eth0": {
                "name": "eth0",
                "network": "lxdbr0",
                "type": "nic"
            },
            "root": {
                "path": "/",
                "pool": "default",
                "type": "disk"
            }
        },
        "name": "testcon",
        "status": "Stopped",
        "status_code": 102,
        "last_used_at": "2020-05-25T06:01:42.421167337Z",
        "location": "none",
        "type": "container"
    } 
DBUG[05-25|06:26:38] Connected to the websocket: ws://unix.socket/1.0/events 
DBUG[05-25|06:26:38] Sending request to LXD                   method=PUT url=http://unix.socket/1.0/instances/testcon/state etag=
DBUG[05-25|06:26:38] 
    {
        "action": "start",
        "timeout": 0,
        "force": false,
        "stateful": false
    } 
DBUG[05-25|06:26:38] Got operation from LXD 
DBUG[05-25|06:26:38] 
    {
        "id": "ed557b66-81e4-4125-b7dd-35ce7244cef6",
        "class": "task",
        "description": "Starting container",
        "created_at": "2020-05-25T06:26:38.665892054Z",
        "updated_at": "2020-05-25T06:26:38.665892054Z",
        "status": "Running",
        "status_code": 103,
        "resources": {
            "containers": [
                "/1.0/containers/testcon"
            ]
        },
        "metadata": null,
        "may_cancel": false,
        "err": "",
        "location": "none"
    } 
DBUG[05-25|06:26:38] Sending request to LXD                   method=GET url=http://unix.socket/1.0/operations/ed557b66-81e4-4125-b7dd-35ce7244cef6 etag=
DBUG[05-25|06:26:38] Got response struct from LXD 
DBUG[05-25|06:26:38] 
    {
        "id": "ed557b66-81e4-4125-b7dd-35ce7244cef6",
        "class": "task",
        "description": "Starting container",
        "created_at": "2020-05-25T06:26:38.665892054Z",
        "updated_at": "2020-05-25T06:26:38.665892054Z",
        "status": "Running",
        "status_code": 103,
        "resources": {
            "containers": [
                "/1.0/containers/testcon"
            ]
        },
        "metadata": null,
        "may_cancel": false,
        "err": "",
        "location": "none"
    }

Error: Cannot apply limits.disk.priority as blkio.weight cgroup controller is missing

Try lxc info --show-log testcon for more info

$ lxc info --show-log testcon
Name: testcon
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/05/24 19:57 UTC
Status: Stopped
Type: container
Profiles: dev, testcon

Log:

lxc testcon 20200525064721.994 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.testcon"
lxc testcon 20200525064721.996 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1143 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.testcon"
lxc testcon 20200525064721.997 ERROR    utils - utils.c:lxc_can_use_pidfd:1855 - Invalid argument - Kernel does not support waiting on processes through pidfds
lxc testcon 20200525064722.111 WARN     cgfsng - cgroups/cgfsng.c:fchowmodat:1455 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )
lxc 20200525064723.504 WARN     commands - commands.c:lxc_cmd_rsp_recv:122 - Connection reset by peer - Failed to receive response for command "get_state"

location: none metadata: class: task created_at: "2020-05-25T06:47:21.598155313Z" description: Starting container err: Cannot apply limits.disk.priority as blkio.weight cgroup controller is missing id: d08cbb6a-d054-47a4-9444-53edd8268b3d location: none may_cancel: false metadata: null resources: containers:

location: none metadata: context: action: stop created: 2020-05-24 19:57:00.248990143 +0000 UTC ephemeral: "false" name: testcon project: default stateful: "false" used: 2020-05-25 06:36:27.679405421 +0000 UTC level: info message: Stopped container timestamp: "2020-05-25T06:47:24.373196634Z" type: logging

brauner commented 4 years ago

Whether or not these controller files are available depends on the io scheduler that is used. And for weight based files it either needs to be CFQ or BFQ. But whether or not enabling them makes sense, depends on your hardware. For NVMe it doesn't make much sense enabling a kernel scheduler as the hardware will just do this itself so it's often turned of by default. You can see this at:

cat /sys/block/sda/queue/scheduler
brunswyck commented 4 years ago

Disk are ordinary SATA-600 disks so no NVMe in my case. I found this on the matter: https://www.kernel.org/doc/html/latest/block/bfq-iosched.html https://stackoverflow.com/questions/1009577/selecting-a-linux-i-o-scheduler So how should I set the IO scheduler for my SATA-600's?

sudo cat /sys/block/sda/queue/scheduler
[mq-deadline] none
stgraber commented 4 years ago

Your system is using SCSI multiqueue which similarly prevents those I/O schedulers from being available. I believe there's a kernel boot option to turn that off which may then get the I/O schedulers that support cgroups to show up again.

stgraber commented 4 years ago

Closing as LXD appears to be behaving as expected. You asked it to configure a kernel feature which isn't available and it fails with a clear error about what's missing.

brunswyck commented 4 years ago

Okay. It would improve documentation if a link with some information regarding io scheduler configuration was provided here under limits.disk.priority I hope this post helps some others. Thank you.

danielrobbins commented 2 years ago

My thoughts on this are that this should be a graceful warning rather than preventing the container from launching. There are too many things in lxd like this and it can make things too fragile in production -- too many 'gotchas' if some underlying configuration changes. (Another example of this is when a storage volume is missing, but it is not being used at all -- lxd will simply fail to start, rather than start in degraded mode or start all containers it can.)

m-ueberall commented 2 years ago

Your system is using SCSI multiqueue which similarly prevents those I/O schedulers from being available. I believe there's a kernel boot option to turn that off which may then get the I/O schedulers that support cgroups to show up again.

@brauner, @stgraber: Is it possible that alternative I/O schedulers like bfq use different controller files instead of /sys/fs/cgroup/blkio/blkio.weight (like /sys/fs/cgroup/blkio/{init.scope,user.slice,system.slice}*/blkio.bfq.weight) that could/should be used instead or am I still missing the first although cat /sys/block/sda/queue/scheduler tells me that bfq should be active (mq-deadline kyber [bfq] none)? (Switching between the schedulers and remounting /sys/fs/cgroup/blkio does not seem to change anything about that.)

stgraber commented 2 years ago

It's certainly possible, though we're very unlikely to be attending support for more files in cgroup1 blkio as the majority of distros have now moved to cgroup2 or are in the process of moving to it. Any new work done on cgroup configuration will be focused on cgroup2 at this point.