canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.38k stars 929 forks source link

Aliases from `lxc image copy --copy-aliases` are exclusive to one image type #14047

Open DanielDewberry opened 2 months ago

DanielDewberry commented 2 months ago

Required information

Distribution: Kubuntu Distribution version: 24.04.1

The output of snap list --all lxd core20 core22 core24 snapd:

Name    Version         Rev    Tracking       Publisher    Notes
core20  20240416        2318   latest/stable  canonical**  base
core22  20240731        1564   latest/stable  canonical**  base,disabled
core22  20240809        1586   latest/stable  canonical**  base
core24  20240528        423    latest/stable  canonical**  base,disabled
core24  20240710        490    latest/stable  canonical**  base
lxd     5.21.2-22f93f4  29948  5.21/stable    canonical**  disabled,held
lxd     5.21.2-2f4ba6b  30131  5.21/stable    canonical**  held
snapd   2.62            21465  latest/stable  canonical**  snapd,disabled
snapd   2.63            21759  latest/stable  canonical**  snapd

The output of lxc info:

config:
  core.https_address: :8443
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- backup_compression
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- instances_nic_host_name
- image_copy_profile
- container_syscall_intercept_sysinfo
- clustering_evacuation_mode
- resources_pci_vpd
- qemu_raw_conf
- storage_cephfs_fscache
- network_load_balancer
- vsock_api
- instance_ready_state
- network_bgp_holdtime
- storage_volumes_all_projects
- metrics_memory_oom_total
- storage_buckets
- storage_buckets_create_credentials
- metrics_cpu_effective_total
- projects_networks_restricted_access
- storage_buckets_local
- loki
- acme
- internal_metrics
- cluster_join_token_expiry
- remote_token_expiry
- init_preseed
- storage_volumes_created_at
- cpu_hotplug
- projects_networks_zones
- network_txqueuelen
- cluster_member_state
- instances_placement_scriptlet
- storage_pool_source_wipe
- zfs_block_mode
- instance_generation_id
- disk_io_cache
- amd_sev
- storage_pool_loop_resize
- migration_vm_live
- ovn_nic_nesting
- oidc
- network_ovn_l3only
- ovn_nic_acceleration_vdpa
- cluster_healing
- instances_state_total
- auth_user
- security_csm
- instances_rebuild
- numa_cpu_placement
- custom_volume_iso
- network_allocations
- storage_api_remote_volume_snapshot_copy
- zfs_delegate
- operations_get_query_all_projects
- metadata_configuration
- syslog_socket
- event_lifecycle_name_and_project
- instances_nic_limits_priority
- disk_initial_volume_configuration
- operation_wait
- cluster_internal_custom_volume_copy
- disk_io_bus
- storage_cephfs_create_missing
- instance_move_config
- ovn_ssl_config
- init_preseed_storage_volumes
- metrics_instances_count
- server_instance_type_info
- resources_disk_mounted
- server_version_lts
- oidc_groups_claim
- loki_config_instance
- storage_volatile_uuid
- import_instance_devices
- instances_uefi_vars
- instances_migration_stateful
- container_syscall_filtering_allow_deny_syntax
- access_management
- vm_disk_io_limits
- storage_volumes_all
- instances_files_modify_permissions
- image_restriction_nesting
- container_syscall_intercept_finit_module
- device_usb_serial
- network_allocate_external_ips
- explicit_trust_token
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
auth_user_name: daniel
auth_user_method: unix
environment:
  addresses:
  - 192.168.1.252:8443
  - 10.12.0.2:8443
  - 10.11.0.2:8443
  - 10.169.240.1:8443
  - '[fd42:a1aa:6705:dc07::1]:8443'
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    MIIB4DCCAWegAwIBAgIRAKn4mCq4VM+D6/mAfQ0Q0GEwCgYIKoZIzj0EAwMwIzEM
    MAoGA1UEChMDTFhEMRMwEQYDVQQDDApyb290QE05MTBxMB4XDTI0MDMyODIyMzg1
    OFoXDTM0MDMyNjIyMzg1OFowIzEMMAoGA1UEChMDTFhEMRMwEQYDVQQDDApyb290
    QE05MTBxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAETygOSLVaV2zcIWfetH1HAl2E
    +G+NFNFxfb20mFnbF90MV8t3k23rEx9Ps3rYysRIqm3VgqPfZCxTycLamWGOU3ob
    pKPIBr1SbEh/HC1nSg9LJjDNgE9BZgRctsJyWYGyo18wXTAOBgNVHQ8BAf8EBAMC
    BaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDAYDVR0TAQH/BAIwADAoBgNVHREEITAf
    ggVNOTEwcYcEfwAAAYcQAAAAAAAAAAAAAAAAAAAAATAKBggqhkjOPQQDAwNnADBk
    AjBGZ1443xO0SdYCI3mEpSQYMR4i3+S0t6rvecBhc/yailhvYbf4/YW4Qr6qjfB4
    jagCMGt+hqyPZIFs/2dEkdTmRk2BEtjomUFjsYoPE7lQc1w4XSFLOtg/sNTZUiMn
    7s/zIw==
    -----END CERTIFICATE-----
  certificate_fingerprint: 2083c2011cce5ea317c4972cb724634f0a2b440b13be2199ef6a13d9bc35b025
  driver: lxc | qemu
  driver_version: 6.0.0 | 8.2.1
  instance_types:
  - container
  - virtual-machine
  firewall: nftables
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
    idmapped_mounts: "true"
    netnsid_getifaddrs: "true"
    seccomp_listener: "true"
    seccomp_listener_continue: "true"
    uevent_injection: "true"
    unpriv_fscaps: "true"
  kernel_version: 6.8.0-41-generic
  lxc_features:
    cgroup2: "true"
    core_scheduling: "true"
    devpts_fd: "true"
    idmapped_mounts_v2: "true"
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    network_veth_router: "true"
    pidfd: "true"
    seccomp_allow_deny_syntax: "true"
    seccomp_notify: "true"
    seccomp_proxy_send_notify_fd: "true"
  os_name: Ubuntu
  os_version: "24.04"
  project: default
  server: lxd
  server_clustered: false
  server_event_mode: full-mesh
  server_name: M910q
  server_pid: 14260
  server_version: 5.21.2
  server_lts: true
  storage: zfs
  storage_version: 2.2.2-0ubuntu9
  storage_supported_drivers:
  - name: cephobject
    version: 17.2.7
    remote: true
  - name: dir
    version: "1"
    remote: false
  - name: lvm
    version: 2.03.11(2) (2021-01-08) / 1.02.175 (2021-01-08) / 4.48.0
    remote: false
  - name: powerflex
    version: 1.16 (nvme-cli)
    remote: true
  - name: zfs
    version: 2.2.2-0ubuntu9
    remote: false
  - name: btrfs
    version: 5.16.2
    remote: false
  - name: ceph
    version: 17.2.7
    remote: true
  - name: cephfs
    version: 17.2.7
    remote: true

Issue description

Copied from https://discourse.ubuntu.com/t/unexpected-behaviour-with-lxc-image-copy-copy-alises/47529?u=dwd-daniel, as requested.

I think that the aliases should be copied to the local storage, and should not be unique across instance types. That is to say, both a VM and Container image should be able to have the jammy or noble alias.

Acquring the images: order 1

Issuing

lxc image copy ubuntu:jammy local: --copy-aliases
lxc image copy ubuntu:jammy local: --copy-aliases --vm
lxc image copy ubuntu:noble local: --copy-aliases
lxc image copy ubuntu:noble local: --copy-aliases --vm

I see the following images are stored:

+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|   ALIAS    | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 | ARCHITECTURE |      TYPE       |   SIZE    |          UPLOAD DATE          |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| j (5 more) | d45dfeba51e0 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 592.69MiB | Aug 26, 2024 at 4:19pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| n (9 more) | 042aedb75f54 | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 558.88MiB | Aug 27, 2024 at 3:32pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | a3a811814328 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 413.98MiB | Aug 25, 2024 at 10:47am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | f0fbf5affa6a | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 241.13MiB | Aug 27, 2024 at 11:10am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+

Notice that the alias is set for the VMs, which are the second of the container/ VM aquisitions.

Acquring the images: order 2

If I switch the order of the Jammy container & VM acquisition, having already executed the previous command:

lxc image copy ubuntu:jammy local: --copy-aliases --vm
lxc image copy ubuntu:jammy local: --copy-aliases
lxc image copy ubuntu:noble local: --copy-aliases
lxc image copy ubuntu:noble local: --copy-aliases --vm

I see this:

+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|   ALIAS    | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 | ARCHITECTURE |      TYPE       |   SIZE    |          UPLOAD DATE          |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| j (5 more) | a3a811814328 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 413.98MiB | Aug 25, 2024 at 10:47am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| n (9 more) | 042aedb75f54 | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 558.88MiB | Aug 27, 2024 at 3:32pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | d45dfeba51e0 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 592.69MiB | Aug 26, 2024 at 4:19pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | f0fbf5affa6a | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 241.13MiB | Aug 27, 2024 at 11:10am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+

The Jammy container image, which was acquired after the Jammy VM image, owns the alias.

Observation vs expectation

The aliases are not copied and preserved for each image, they seem to be required to one image, irrespective of image type. I expected each image to retain its aliases, and the --vm switch to be used as a way to discerne between the container and VMs images.

We can see the aliases exist on the remote:

$ lxc image info ubuntu:jammy

Fingerprint: a3a8118143289e285ec44b489fb1a0811da75c27a22004f7cd34db70a60a0af4
Size: 413.98MiB
Architecture: x86_64
Type: container
Public: yes
Timestamps:
    Created: 2024/08/21 00:00 UTC
    Uploaded: 2024/08/21 00:00 UTC
    Expires: 2027/06/01 00:00 UTC
    Last used: never
Properties:
    serial: 20240821
    description: ubuntu 22.04 LTS amd64 (release) (20240821)
    type: squashfs
    os: ubuntu
    release: jammy
    version: 22.04
    architecture: amd64
    label: release
Aliases:
    - 22.04
    - 22.04/amd64
    - j
    - j/amd64
    - jammy
    - jammy/amd64
Cached: no
Auto update: disabled
Profiles: []
$ lxc image info ubuntu:jammy --vm

Fingerprint: d45dfeba51e0008dbd0bb0828a2684b54274aabde43bbf9fa4db68d2bd97813e
Size: 592.69MiB
Architecture: x86_64
Type: virtual-machine
Public: yes
Timestamps:
    Created: 2024/08/21 00:00 UTC
    Uploaded: 2024/08/21 00:00 UTC
    Expires: 2027/06/01 00:00 UTC
    Last used: never
Properties:
    serial: 20240821
    description: ubuntu 22.04 LTS amd64 (release) (20240821)
    type: disk-kvm.img
    os: ubuntu
    release: jammy
    version: 22.04
    architecture: amd64
    label: release
Aliases:
    - 22.04
    - 22.04/amd64
    - j
    - j/amd64
    - jammy
    - jammy/amd64
Cached: no
Auto update: disabled
Profiles: []
$ lxc image info ubuntu:noble 

Fingerprint: f0fbf5affa6a3c22848f24ba304525354023fdc33f4390aa816ae05bdb94dbe9
Size: 241.13MiB
Architecture: x86_64
Type: container
Public: yes
Timestamps:
    Created: 2024/08/21 00:00 UTC
    Uploaded: 2024/08/21 00:00 UTC
    Expires: 2029/05/31 00:00 UTC
    Last used: never
Properties:
    serial: 20240821
    description: ubuntu 24.04 LTS amd64 (release) (20240821)
    type: squashfs
    os: ubuntu
    release: noble
    version: 24.04
    architecture: amd64
    label: release
Aliases:
    - 24.04
    - 24.04/amd64
    - n
    - n/amd64
    - noble
    - noble/amd64
    - lts
    - lts/amd64
    - default
    - default/amd64
Cached: no
Auto update: disabled
Profiles: []
$ lxc image info ubuntu:noble --vm

Fingerprint: 042aedb75f54341bc8e16f737e2acec7a1561b1dfd4502cb09398a4a36549eb7
Size: 558.88MiB
Architecture: x86_64
Type: virtual-machine
Public: yes
Timestamps:
    Created: 2024/08/21 00:00 UTC
    Uploaded: 2024/08/21 00:00 UTC
    Expires: 2029/05/31 00:00 UTC
    Last used: never
Properties:
    os: ubuntu
    release: noble
    version: 24.04
    architecture: amd64
    label: release
    serial: 20240821
    description: ubuntu 24.04 LTS amd64 (release) (20240821)
    type: disk1.img
Aliases:
    - 24.04
    - 24.04/amd64
    - n
    - n/amd64
    - noble
    - noble/amd64
    - lts
    - lts/amd64
    - default
    - default/amd64
Cached: no
Auto update: disabled
Profiles: []

Attempting to set local aliases

If I attempt to add an alias to the local copy of the Jammy container, I am informed the Jammy alias already exists, which leads me to believe the alias must be, as previously noted, unique to a single image irrespective of the image type.

$ lxc image alias create jammy a3a811814328

Error: Alias "jammy" already exists

$ lxc image list
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|   ALIAS    | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 | ARCHITECTURE |      TYPE       |   SIZE    |          UPLOAD DATE          |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| j (5 more) | d45dfeba51e0 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 592.69MiB | Aug 26, 2024 at 4:19pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| n (9 more) | 042aedb75f54 | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 558.88MiB | Aug 27, 2024 at 3:32pm (UTC)  |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | a3a811814328 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 413.98MiB | Aug 25, 2024 at 10:47am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|            | f0fbf5affa6a | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 241.13MiB | Aug 27, 2024 at 11:10am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+

I can set up my own aliases which correctly categorize the images:

$ lxc image alias create local:jammy-ctr a3a811814328
$ lxc image alias create local:jammy-vm  d45dfeba51e0
$ lxc image alias create local:noble-ctr f0fbf5affa6a
$ lxc image alias create local:noble-vm  042aedb75f54
$ lxc image list

+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
|    ALIAS    | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 | ARCHITECTURE |      TYPE       |   SIZE    |          UPLOAD DATE          |
+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| j (6 more)  | d45dfeba51e0 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 592.69MiB | Aug 26, 2024 at 4:19pm (UTC)  |
+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| jammy-ctr   | a3a811814328 | no     | ubuntu 22.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 413.98MiB | Aug 25, 2024 at 10:47am (UTC) |
+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| n (10 more) | 042aedb75f54 | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | VIRTUAL-MACHINE | 558.88MiB | Aug 27, 2024 at 3:32pm (UTC)  |
+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+
| noble-ctr   | f0fbf5affa6a | no     | ubuntu 24.04 LTS amd64 (release) (20240821) | x86_64       | CONTAINER       | 241.13MiB | Aug 27, 2024 at 11:10am (UTC) |
+-------------+--------------+--------+---------------------------------------------+--------------+-----------------+-----------+-------------------------------+

Summary

I think that the aliases should be copied to the local storage, and should not be unique across instance types. That is to say, both a VM and Container image should be able to have the jammy or noble alias.

Best regards

Daniel

kadinsayani commented 2 months ago

The issue is in the images_aliases table. When a POST request is made to /1.0/images/aliases, the image_id column is overwritten.

kadinsayani@devbox:~/canonical/lxd$ lxc image copy ubuntu:jammy local: --copy-aliases --vm
Image copied successfully!
kadinsayani@devbox:~/canonical/lxd$ lxd sql global 'select * from images_aliases'
+----+-------------+----------+-------------+------------+
| id |    name     | image_id | description | project_id |
+----+-------------+----------+-------------+------------+
| 31 | 22.04       | 15       |             | 1          |
| 32 | 22.04/amd64 | 15       |             | 1          |
| 33 | j           | 15       |             | 1          |
| 34 | j/amd64     | 15       |             | 1          |
| 35 | jammy       | 15       |             | 1          |
| 36 | jammy/amd64 | 15       |             | 1          |
+----+-------------+----------+-------------+------------+
kadinsayani@devbox:~/canonical/lxd$ lxc image copy ubuntu:jammy local: --copy-aliases
Image copied successfully!
kadinsayani@devbox:~/canonical/lxd$ lxd sql global 'select * from images_aliases'
+----+-------------+----------+-------------+------------+
| id |    name     | image_id | description | project_id |
+----+-------------+----------+-------------+------------+
| 37 | 22.04       | 10       |             | 1          |
| 38 | 22.04/amd64 | 10       |             | 1          |
| 39 | j           | 10       |             | 1          |
| 40 | j/amd64     | 10       |             | 1          |
| 41 | jammy       | 10       |             | 1          |
| 42 | jammy/amd64 | 10       |             | 1          |
+----+-------------+----------+-------------+------------+

I see two options forward:

1) Separate images_aliases into two distinct tables, container_images_aliases and vm_images_aliases; or 2) Separate the image_id column into two distinct columns, container_image_id and vm_image_id (less overhead with this option).

In either case, an alias with the same name could belong to both a container and VM.

Which option would be preferable? cc. @tomponline @hamistao

tomponline commented 2 months ago

The issue is in the images_aliases table. When a POST request is made to /1.0/images/aliases, the image_id column is overwritten.

kadinsayani@devbox:~/canonical/lxd$ lxc image copy ubuntu:jammy local: --copy-aliases --vm
Image copied successfully!
kadinsayani@devbox:~/canonical/lxd$ lxd sql global 'select * from images_aliases'
+----+-------------+----------+-------------+------------+
| id |    name     | image_id | description | project_id |
+----+-------------+----------+-------------+------------+
| 31 | 22.04       | 15       |             | 1          |
| 32 | 22.04/amd64 | 15       |             | 1          |
| 33 | j           | 15       |             | 1          |
| 34 | j/amd64     | 15       |             | 1          |
| 35 | jammy       | 15       |             | 1          |
| 36 | jammy/amd64 | 15       |             | 1          |
+----+-------------+----------+-------------+------------+
kadinsayani@devbox:~/canonical/lxd$ lxc image copy ubuntu:jammy local: --copy-aliases
Image copied successfully!
kadinsayani@devbox:~/canonical/lxd$ lxd sql global 'select * from images_aliases'
+----+-------------+----------+-------------+------------+
| id |    name     | image_id | description | project_id |
+----+-------------+----------+-------------+------------+
| 37 | 22.04       | 10       |             | 1          |
| 38 | 22.04/amd64 | 10       |             | 1          |
| 39 | j           | 10       |             | 1          |
| 40 | j/amd64     | 10       |             | 1          |
| 41 | jammy       | 10       |             | 1          |
| 42 | jammy/amd64 | 10       |             | 1          |
+----+-------------+----------+-------------+------------+

I see two options forward:

1. Separate `images_aliases` into two distinct tables, `container_images_aliases` and `vm_images_aliases`; or

2. Separate the `image_id` column into two distinct columns, `container_image_id` and `vm_image_id` (less overhead with this option).

In either case, an alias with the same name could belong to both a container and VM.

Which option would be preferable? cc. @tomponline @hamistao

is name a unique index?

tomponline commented 2 months ago

@kadinsayani does LXD auto select the correct instance type if launch an instance using an image alias of type VM without specifying --vm?

kadinsayani commented 2 months ago

@kadinsayani does LXD auto select the correct instance type if launch an instance using an image alias of type VM without specifying --vm?

Yes, applyAliases() in /shared/simplestreams/simplestreams.go fetches all images and applies aliases by filling a slice of alias structs with properties from api.Image structs, which include an image type.

kadinsayani commented 2 months ago

is name a unique index?

Yes. We could also change name to not be a unique index, and rely on a unique id.

Edit: I don't think it's possible to remove a unique constraint from a table. This approach would require creating a new table with different constraints and copying everything over.

hamistao commented 2 months ago

Yes. We could also change name to not be a unique index, and rely on a unique id.

This would be best in my view, since it would be a simpler change and wouldn't break any (I think) lxd sql queries currently being used.

Edit: @kadinsayani independentely of which approach we choose, I think you should be able to fix this by patching LXD: https://github.com/canonical/lxd/blob/5c3440f95557581119ffaba36622a6e1efe91e96/lxd/patches.go#L1437-L1439

tomponline commented 2 months ago

So does this issue boil down to:

Simplestreams aliases are unique per instance type, but LXD's image aliases are unique (excluding instance type)?

My worry here is that changing the behavior could be considered a breaking API change.

Perhaps rather than just re-assigning the aliases we should error out and let the user decide which aliases they want to keep/update.

DanielDewberry commented 2 months ago

Thank you all looking into this issue.

My worry here is that changing the behavior could be considered a breaking API change.

@tomponline As the user, I consider the CLI API behaviour to be misleading/ incorrect because the expectation is that lxc launch local:jammy (without --vm) should only ever launch a container, but it will actually launch a VM if the alias is assigned to a VM.

Unique alias per image type is the solution I think best aligns with the documentation, but I appreciate your point and suggest that if you follow your approach the above mentioned CLI behaviour be accounted for, e.g.:

$ lxc launch local:jammy 

E: You tried to create a container using the "local" alias "jammy", 
which doesn't exist. The "local" alias "jammy" does 
exist for a VM image."
$ lxc launch local:jammy --vm

E: You tried to create a VM using the "local" alias "jammy", 
which doesn't exist. The "local" alias "jammy" does 
exist for a container image."

That or the documentation be updated to reflect the possibility that the alias may cause the "wrong" kind of instance to be launched.

Best regards

Daniel

kadinsayani commented 1 month ago

Upon further review of the issue, the changes required to resolve this issue are as follows:

API

DB

LXC

For reference: https://github.com/canonical/lxd/pull/14162.

kadinsayani commented 1 week ago

Hi @DanielDewberry,

Just wanted to let you know that this is still on my radar. I'm hoping I'll be able to resume working on it soon.

Best regards, Kadin