derrickmehaffy commented 2 years ago

Required information

Distribution: Linux Mint (Client) // Ubuntu (Server)
Distribution version: 20.3 (Client) // 20.04 (Server)
The output of "lxc info" or if that fails:
lxc info output

config: cluster.https_address: 192.168.15.23:8443 core.https_address: '[::]' core.trust_password: true api_extensions:

storage_zfs_remove_snapshots

container_host_shutdown_timeout

container_stop_priority

container_syscall_filtering

auth_pki

container_last_used_at

etag

patch

usb_devices

https_allowed_credentials

image_compression_algorithm

directory_manipulation

container_cpu_time

storage_zfs_use_refquota

storage_lvm_mount_options

network

profile_usedby

container_push

container_exec_recording

certificate_update

container_exec_signal_handling

gpu_devices

container_image_properties

migration_progress

id_map

network_firewall_filtering

network_routes

storage

file_delete

file_append

network_dhcp_expiry

storage_lvm_vg_rename

storage_lvm_thinpool_rename

network_vlan

image_create_aliases

container_stateless_copy

container_only_migration

storage_zfs_clone_copy

unix_device_rename

storage_lvm_use_thinpool

storage_rsync_bwlimit

network_vxlan_interface

storage_btrfs_mount_options

entity_description

image_force_refresh

storage_lvm_lv_resizing

id_map_base

file_symlinks

container_push_target

network_vlan_physical

storage_images_delete

container_edit_metadata

container_snapshot_stateful_migration

storage_driver_ceph

storage_ceph_user_name

resource_limits

storage_volatile_initial_source

storage_ceph_force_osd_reuse

storage_block_filesystem_btrfs

resources

kernel_limits

storage_api_volume_rename

macaroon_authentication

network_sriov

console

restrict_devlxd

migration_pre_copy

infiniband

maas_network

devlxd_events

proxy

network_dhcp_gateway

file_get_symlink

network_leases

unix_device_hotplug

storage_api_local_volume_handling

operation_description

clustering

event_lifecycle

storage_api_remote_volume_handling

nvidia_runtime

container_mount_propagation

container_backup

devlxd_images

container_local_cross_pool_handling

proxy_unix

proxy_udp

clustering_join

proxy_tcp_udp_multi_port_handling

network_state

proxy_unix_dac_properties

container_protection_delete

unix_priv_drop

pprof_http

proxy_haproxy_protocol

network_hwaddr

proxy_nat

network_nat_order

container_full

candid_authentication

backup_compression

candid_config

nvidia_runtime_config

storage_api_volume_snapshots

storage_unmapped

projects

candid_config_key

network_vxlan_ttl

container_incremental_copy

usb_optional_vendorid

snapshot_scheduling

snapshot_schedule_aliases

container_copy_project

clustering_server_address

clustering_image_replication

container_protection_shift

snapshot_expiry

container_backup_override_pool

snapshot_expiry_creation

network_leases_location

resources_cpu_socket

resources_gpu

resources_numa

kernel_features

id_map_current

event_location

storage_api_remote_volume_snapshots

network_nat_address

container_nic_routes

rbac

cluster_internal_copy

seccomp_notify

lxc_features

container_nic_ipvlan

network_vlan_sriov

storage_cephfs

container_nic_ipfilter

resources_v2

container_exec_user_group_cwd

container_syscall_intercept

container_disk_shift

storage_shifted

resources_infiniband

daemon_storage

instances

image_types

resources_disk_sata

clustering_roles

images_expiry

resources_network_firmware

backup_compression_algorithm

ceph_data_pool_name

container_syscall_intercept_mount

compression_squashfs

container_raw_mount

container_nic_routed

container_syscall_intercept_mount_fuse

container_disk_ceph

virtual-machines

image_profiles

clustering_architecture

resources_disk_id

storage_lvm_stripes

vm_boot_priority

unix_hotplug_devices

api_filtering

instance_nic_network

clustering_sizing

firewall_driver

projects_limits

container_syscall_intercept_hugetlbfs

limits_hugepages

container_nic_routed_gateway

projects_restrictions

custom_volume_snapshot_expiry

volume_snapshot_scheduling

trust_ca_certificates

snapshot_disk_usage

clustering_edit_roles

container_nic_routed_host_address

container_nic_ipvlan_gateway

resources_usb_pci

resources_cpu_threads_numa

resources_cpu_core_die

api_os

container_nic_routed_host_table

container_nic_ipvlan_host_table

container_nic_ipvlan_mode

resources_system

images_push_relay

network_dns_search

container_nic_routed_limits

instance_nic_bridged_vlan

network_state_bond_bridge

usedby_consistency

custom_block_volumes

clustering_failure_domains

resources_gpu_mdev

console_vga_type

projects_limits_disk

network_type_macvlan

network_type_sriov

container_syscall_intercept_bpf_devices

network_type_ovn

projects_networks

projects_networks_restricted_uplinks

custom_volume_backup

backup_override_name

storage_rsync_compression

network_type_physical

network_ovn_external_subnets

network_ovn_nat

network_ovn_external_routes_remove

tpm_device_type

storage_zfs_clone_copy_rebase

gpu_mdev

resources_pci_iommu

resources_network_usb

resources_disk_address

network_physical_ovn_ingress_mode

network_ovn_dhcp

network_physical_routes_anycast

projects_limits_instances

network_state_vlan

instance_nic_bridged_port_isolation

instance_bulk_state_change

network_gvrp

instance_pool_move

gpu_sriov

pci_device_type

storage_volume_state

network_acl

migration_stateful

disk_state_quota

storage_ceph_features

projects_compression

projects_images_remote_cache_expiry

certificate_project

network_ovn_acl

projects_images_auto_update

projects_restricted_cluster_target

images_default_architecture

network_ovn_acl_defaults

gpu_mig

project_usage

network_bridge_acl

warnings

projects_restricted_backups_and_snapshots

clustering_join_token

clustering_description

server_trusted_proxy

clustering_update_cert

storage_api_project

server_instance_driver_operational

server_supported_storage_drivers

event_lifecycle_requestor_address

resources_gpu_usb

clustering_evacuation

network_ovn_nat_address

network_bgp

network_forward

custom_volume_refresh

network_counters_errors_dropped

metrics

image_source_project

clustering_config

network_peer

linux_sysctl

network_dns

ovn_nic_acceleration

certificate_self_renewal

instance_project_move

storage_volume_project_move

cloud_init

network_dns_nat

database_leader

instance_all_projects

clustering_groups

ceph_rbd_du

instance_get_full

qemu_metrics

gpu_mig_uuid

event_project

clustering_evacuation_live

instance_allow_inconsistent_copy api_status: stable api_version: "1.0" auth: trusted public: false auth_methods:

tls environment: addresses:

192.168.15.23:8443

'[fdb3:89cd:f82f:4e1b:1844:8cff:fec5:b029]:8443'

'[fdb3:89cd:f82f:4e1b:6e0b:84ff:fee0:667d]:8443'

'[fdb3:89cd:f82f:4e1b:509d:b8ff:fe4a:78bb]:8443'

240.23.0.1:8443 architectures:

x86_64

i686 certificate: | -----BEGIN CERTIFICATE----- MIIB/jCCAYSgAwIBAgIRAJT/dZ2R8ni+2Y5RS8F5IJAwCgYIKoZIzj0EAwMwMjEc MBoGA1UEChMTbGludXhjb250YWluZXJzLm9yZzESMBAGA1UEAwwJcm9vdEBsY2gx MB4XDTIyMDExNzAwMzg0OVoXDTMyMDExNTAwMzg0OVowMjEcMBoGA1UEChMTbGlu dXhjb250YWluZXJzLm9yZzESMBAGA1UEAwwJcm9vdEBsY2gxMHYwEAYHKoZIzj0C AQYFK4EEACIDYgAEy3Lvo06AqDkTGuKG8EAxwm2nY2CV8mpEswowO++6oTVshY0e Juxpi7nY2oCoDuHPhi/06Hhp74wL5Fd8Fgp5TsMUSAYH3ECwe/+hWzi1wMcSRswo uLs3RuraUS4mictgo14wXDAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYB BQUHAwEwDAYDVR0TAQH/BAIwADAnBgNVHREEIDAeggRsY2gxhwR/AAABhxAAAAAA AAAAAAAAAAAAAAABMAoGCCqGSM49BAMDA2gAMGUCME2oeDhMLosXdo0HO/w9cGNR CkwVL+GrdDW2f3s3V9D023liA9F3vtTWwGZxmDCfawIxAJ3fAlOjyycB9vFGN/6s vBiuAZEtXiQDXq0MgPRf4H0kJIF+8mZuGoiI86kl3QW0yw== -----END CERTIFICATE----- certificate_fingerprint: 96a474409e231b279577b2ec071e30ffee09ceddcd5909eb0a9a0dff328d66d2 driver: lxc driver_version: 4.0.11 firewall: nftables kernel: Linux kernel_architecture: x86_64 kernel_features: netnsid_getifaddrs: "true" seccomp_listener: "true" seccomp_listener_continue: "true" shiftfs: "false" uevent_injection: "true" unpriv_fscaps: "true" kernel_version: 5.4.0-96-generic lxc_features: cgroup2: "true" core_scheduling: "true" devpts_fd: "true" idmapped_mounts_v2: "true" mount_injection_file: "true" network_gateway_device_route: "true" network_ipvlan: "true" network_l2proxy: "true" network_phys_macvlan_mtu: "true" network_veth_router: "true" pidfd: "true" seccomp_allow_deny_syntax: "true" seccomp_notify: "true" seccomp_proxy_send_notify_fd: "true" os_name: Ubuntu os_version: "20.04" project: default server: lxd server_clustered: true server_name: lch1 server_pid: 1110 server_version: "4.22" storage: zfs storage_version: 0.8.3-1ubuntu12.13 storage_supported_drivers:

name: btrfs version: 5.4.1 remote: false

name: cephfs version: 15.2.14 remote: true

name: dir version: "1" remote: false

name: lvm version: 2.03.07(2) (2019-11-30) / 1.02.167 (2019-11-30) / 4.41.0 remote: false

name: zfs version: 0.8.3-1ubuntu12.13 remote: false

name: ceph version: 15.2.14 remote: true

Issue description

I have a 3 node LXD Cluster setup:

And I currently have 14 containers spread ~evenly between all 3 nodes of the cluster:

Each node has a storage pool configured called local-m2 (this is the only storage on each node so all profiles, containers, and images are on this storage pool).

When I run lxc storage list the count provided this command under used by is wrong and it appears to only be taking the count from one node of the cluster. It doesn't appear that I can pass the --target into this command either, I have to use the lxc storage info --target <node-name> local-m2 in order to get the actual list of containers in each node's storage pool.

Expected behavior

I believe it's intended that a simple lxc storage list should take the used by information from all members of the cluster as the storage pool is configured at the cluster level.

Information to attach

[ ] Any relevant kernel output (dmesg)
[ ] Container log (lxc info NAME --show-log)
[ ] Container configuration (lxc config show NAME --expanded)
[ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log)
[ ] Output of the client with --debug
[ ] Output of the daemon with --debug (alternatively output of lxc monitor while reproducing the issue)

derrickmehaffy commented 2 years ago

Let me know if this is a duplicate, I tried searching other issues here and on the lxc repo / forum and didn't see anything that really described this.

Also not entirely sure if this is intended or not (I'm a bit new to lxc/lxd) or if this is even the right repo :sweat_smile: to report this issue on. If you need any other information let me know!

stgraber commented 2 years ago

@markylaing want to take a stab at this one?

I think it's right that the UsedBy on a storage pool should reflect volumes on all cluster members. The URLs will need to include the target=NAME field though so we don't end up with duplicates, unless we're dealing with a remote storage pool that is, in which case volume should only be listed once.

derrickmehaffy commented 2 years ago

@markylaing want to take a stab at this one?

I think it's right that the UsedBy on a storage pool should reflect volumes on all cluster members. The URLs will need to include the target=NAME field though so we don't end up with duplicates, unless we're dealing with a remote storage pool that is, in which case volume should only be listed once.

Just to reinforce this, some other examples of this, the profiles show the proper count across the cluster:

(Also thanks for answering so quickly :star_struck: )

derrickmehaffy commented 2 years ago

Holy cow that was fast, thanks @markylaing & @stgraber \o/

canonical / lxd

Cluster - storage list provides wrong used by count #9842

Required information

Issue description

Expected behavior

Information to attach