Daemon crash on vfsDatabaseRead, database corruption

[project0]

Required information

• Distribution: arch-linux
• Distribution version: latest patches
• The output of "incus info" or if that fails:
  • Kernel version: 6.9.10-arch1-1
  • LXC version: -
  • Incus version: 6.3
  • Storage backend in use: btrfs

Issue description

Similar issue to #665, see debug log. It happened now the second time to me.

vfsDatabaseRead: Assertion `amount == (int)page_size' failed.

Only fix i could find is wiping the full sqlite database...

Steps to reproduce

I am not sure how to reproduce it, it seem to happen randomly without after creating or modifying any profile or VM/Container.

Information to attach

• [ ] Any relevant kernel output (dmesg) No kernel error message
• [ ] Container log (incus info NAME --show-log)
• [ ] Container configuration (incus config show NAME --expanded)
• [ ] Main daemon log (at /var/log/incus/incusd.log)
• [ ] Output of the client with --debug
• [x] Output of the daemon with --debug (alternatively output of incus monitor --pretty while reproducing the issue) Debug Log:
  Details

Jul 20 13:51:00 stargazer systemd[1]: Started Incus Container Hypervisor.
Jul 20 13:51:00 stargazer incusd[1083]: time="2024-07-20T13:51:00+02:00" level=debug msg="Success for operation" class=task description="Updating instance types" operation=20c4f101-59b3-4217-a48e-2a1ce55c5930 project=
Jul 20 13:51:00 stargazer incusd[1083]: time="2024-07-20T13:51:00+02:00" level=info msg="Done updating instance types"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url=/1.0/projects/default username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url=/1.0 username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {\n\t\t\t\"config\": {\n\t\t\t\t\"core.https_address\": \":8443\",\n\t\t\t\t\"core.storage_buckets_address\": \":8555\"\n\t\t\t},\n\t\t\t\"api_extensions\": [\n\t\t\t\t\"storage_zfs_remove_snapshots\",\n\t\t\t\t\"container_host_shutdown_timeout\",\n\t\t\t\t\"container_stop_priority\",\n\t\t\t\t\"container_syscall_filtering\",\n\t\t\t\t\"auth_pki\",\n\t\t\t\t\"container_last_used_at\",\n\t\t\t\t\"etag\",\n\t\t\t\t\"patch\",\n\t\t\t\t\"usb_devices\",\n\t\t\t\t\"https_allowed_credentials\",\n\t\t\t\t\"image_compression_algorithm\",\n\t\t\t\t\"directory_manipulation\",\n\t\t\t\t\"container_cpu_time\",\n\t\t\t\t\"storage_zfs_use_refquota\",\n\t\t\t\t\"storage_lvm_mount_options\",\n\t\t\t\t\"network\",\n\t\t\t\t\"profile_usedby\",\n\t\t\t\t\"container_push\",\n\t\t\t\t\"container_exec_recording\",\n\t\t\t\t\"certificate_update\",\n\t\t\t\t\"container_exec_signal_handling\",\n\t\t\t\t\"gpu_devices\",\n\t\t\t\t\"container_image_properties\",\n\t\t\t\t\"migration_progress\",\n\t\t\t\t\"id_map\",\n\t\t\t\t\"network_firewall_filtering\",\n\t\t\t\t\"network_routes\",\n\t\t\t\t\"storage\",\n\t\t\t\t\"file_delete\",\n\t\t\t\t\"file_append\",\n\t\t\t\t\"network_dhcp_expiry\",\n\t\t\t\t\"storage_lvm_vg_rename\",\n\t\t\t\t\"storage_lvm_thinpool_rename\",\n\t\t\t\t\"network_vlan\",\n\t\t\t\t\"image_create_aliases\",\n\t\t\t\t\"container_stateless_copy\",\n\t\t\t\t\"container_only_migration\",\n\t\t\t\t\"storage_zfs_clone_copy\",\n\t\t\t\t\"unix_device_rename\",\n\t\t\t\t\"storage_lvm_use_thinpool\",\n\t\t\t\t\"storage_rsync_bwlimit\",\n\t\t\t\t\"network_vxlan_interface\",\n\t\t\t\t\"storage_btrfs_mount_options\",\n\t\t\t\t\"entity_description\",\n\t\t\t\t\"image_force_refresh\",\n\t\t\t\t\"storage_lvm_lv_resizing\",\n\t\t\t\t\"id_map_base\",\n\t\t\t\t\"file_symlinks\",\n\t\t\t\t\"container_push_target\",\n\t\t\t\t\"network_vlan_physical\",\n\t\t\t\t\"storage_images_delete\",\n\t\t\t\t\"container_edit_metadata\",\n\t\t\t\t\"container_snapshot_stateful_migration\",\n\t\t\t\t\"storage_driver_ceph\",\n\t\t\t\t\"storage_ceph_user_name\",\n\t\t\t\t\"resource_limits\",\n\t\t\t\t\"storage_volatile_initial_source\",\n\t\t\t\t\"storage_ceph_force_osd_reuse\",\n\t\t\t\t\"storage_block_filesystem_btrfs\",\n\t\t\t\t\"resources\",\n\t\t\t\t\"kernel_limits\",\n\t\t\t\t\"storage_api_volume_rename\",\n\t\t\t\t\"network_sriov\",\n\t\t\t\t\"console\",\n\t\t\t\t\"restrict_dev_incus\",\n\t\t\t\t\"migration_pre_copy\",\n\t\t\t\t\"infiniband\",\n\t\t\t\t\"dev_incus_events\",\n\t\t\t\t\"proxy\",\n\t\t\t\t\"network_dhcp_gateway\",\n\t\t\t\t\"file_get_symlink\",\n\t\t\t\t\"network_leases\",\n\t\t\t\t\"unix_device_hotplug\",\n\t\t\t\t\"storage_api_local_volume_handling\",\n\t\t\t\t\"operation_description\",\n\t\t\t\t\"clustering\",\n\t\t\t\t\"event_lifecycle\",\n\t\t\t\t\"storage_api_remote_volume_handling\",\n\t\t\t\t\"nvidia_runtime\",\n\t\t\t\t\"container_mount_propagation\",\n\t\t\t\t\"container_backup\",\n\t\t\t\t\"dev_incus_images\",\n\t\t\t\t\"container_local_cross_pool_handling\",\n\t\t\t\t\"proxy_unix\",\n\t\t\t\t\"proxy_udp\",\n\t\t\t\t\"clustering_join\",\n\t\t\t\t\"proxy_tcp_udp_multi_port_handling\",\n\t\t\t\t\"network_state\",\n\t\t\t\t\"proxy_unix_dac_properties\",\n\t\t\t\t\"container_protection_delete\",\n\t\t\t\t\"unix_priv_drop\",\n\t\t\t\t\"pprof_http\",\n\t\t\t\t\"proxy_haproxy_protocol\",\n\t\t\t\t\"network_hwaddr\",\n\t\t\t\t\"proxy_nat\",\n\t\t\t\t\"network_nat_order\",\n\t\t\t\t\"container_full\",\n\t\t\t\t\"backup_compression\",\n\t\t\t\t\"nvidia_runtime_config\",\n\t\t\t\t\"storage_api_volume_snapshots\",\n\t\t\t\t\"storage_unmapped\",\n\t\t\t\t\"projects\",\n\t\t\t\t\"network_vxlan_ttl\",\n\t\t\t\t\"container_incremental_copy\",\n\t\t\t\t\"usb_optional_vendorid\",\n\t\t\t\t\"snapshot_scheduling\",\n\t\t\t\t\"snapshot_schedule_aliases\",\n\t\t\t\t\"container_copy_project\",\n\t\t\t\t\"clustering_server_address\",\n\t\t\t\t\"clustering_image_replication\",\n\t\t\t\t\"container_protection_shift\",\n\t\t\t\t\"snapshot_expiry\",\n\t\t\t\t\"container_backup_override_pool\",\n\t\t\t\t\"snapshot_expiry_creation\",\n\t\t\t\t\"network_leases_location\",\n\t\t\t\t\"resources_cpu_socket\",\n\t\t\t\t\"resources_gpu\",\n\t\t\t\t\"resources_numa\",\n\t\t\t\t\"kernel_features\",\n\t\t\t\t\"id_map_current\",\n\t\t\t\t\"event_location\",\n\t\t\t\t\"storage_api_remote_volume_snapshots\",\n\t\t\t\t\"network_nat_address\",\n\t\t\t\t\"container_nic_routes\",\n\t\t\t\t\"cluster_internal_copy\",\n\t\t\t\t\"seccomp_notify\",\n\t\t\t\t\"lxc_features\",\n\t\t\t\t\"container_nic_ipvlan\",\n\t\t\t\t\"network_vlan_sriov\",\n\t\t\t\t\"storage_cephfs\",\n\t\t\t\t\"container_nic_ipfilter\",\n\t\t\t\t\"resources_v2\",\n\t\t\t\t\"container_exec_user_group_cwd\",\n\t\t\t\t\"container_syscall_intercept\",\n\t\t\t\t\"container_disk_shift\",\n\t\t\t\t\"storage_shifted\",\n\t\t\t\t\"resources_infiniband\",\n\t\t\t\t\"daemon_storage\",\n\t\t\t\t\"instances\",\n\t\t\t\t\"image_types\",\n\t\t\t\t\"resources_disk_sata\",\n\t\t\t\t\"clustering_roles\",\n\t\t\t\t\"images_expiry\",\n\t\t\t\t\"resources_network_firmware\",\n\t\t\t\t\"backup_compression_algorithm\",\n\t\t\t\t\"ceph_data_pool_name\",\n\t\t\t\t\"container_syscall_intercept_mount\",\n\t\t\t\t\"compression_squashfs\",\n\t\t\t\t\"container_raw_mount\",\n\t\t\t\t\"container_nic_routed\",\n\t\t\t\t\"container_syscall_intercept_mount_fuse\",\n\t\t\t\t\"container_disk_ceph\",\n\t\t\t\t\"virtual-machines\",\n\t\t\t\t\"image_profiles\",\n\t\t\t\t\"clustering_architecture\",\n\t\t\t\t\"resources_disk_id\",\n\t\t\t\t\"storage_lvm_stripes\",\n\t\t\t\t\"vm_boot_priority\",\n\t\t\t\t\"unix_hotplug_devices\",\n\t\t\t\t\"api_filtering\",\n\t\t\t\t\"instance_nic_network\",\n\t\t\t\t\"clustering_sizing\",\n\t\t\t\t\"firewall_driver\",\n\t\t\t\t\"projects_limits\",\n\t\t\t\t\"container_syscall_intercept_hugetlbfs\",\n\t\t\t\t\"limits_hugepages\",\n\t\t\t\t\"container_nic_routed_gateway\",\n\t\t\t\t\"projects_restrictions\",\n\t\t\t\t\"custom_volume_snapshot_expiry\",\n\t\t\t\t\"volume_snapshot_scheduling\",\n\t\t\t\t\"trust_ca_certificates\",\n\t\t\t\t\"snapshot_disk_usage\",\n\t\t\t\t\"clustering_edit_roles\",\n\t\t\t\t\"container_nic_routed_host_address\",\n\t\t\t\t\"container_nic_ipvlan_gateway\",\n\t\t\t\t\"resources_usb_pci\",\n\t\t\t\t\"resources_cpu_threads_numa\",\n\t\t\t\t\"resources_cpu_core_die\",\n\t\t\t\t\"api_os\",\n\t\t\t\t\"container_nic_routed_host_table\",\n\t\t\t\t\"container_nic_ipvlan_host_table\",\n\t\t\t\t\"container_nic_ipvlan_mode\",\n\t\t\t\t\"resources_system\",\n\t\t\t\t\"images_push_relay\",\n\t\t\t\t\"network_dns_search\",\n\t\t\t\t\"container_nic_routed_limits\",\n\t\t\t\t\"instance_nic_bridged_vlan\",\n\t\t\t\t\"network_state_bond_bridge\",\n\t\t\t\t\"usedby_consistency\",\n\t\t\t\t\"custom_block_volumes\",\n\t\t\t\t\"clustering_failure_domains\",\n\t\t\t\t\"resources_gpu_mdev\",\n\t\t\t\t\"console_vga_type\",\n\t\t\t\t\"projects_limits_disk\",\n\t\t\t\t\"network_type_macvlan\",\n\t\t\t\t\"network_type_sriov\",\n\t\t\t\t\"container_syscall_intercept_bpf_devices\",\n\t\t\t\t\"network_type_ovn\",\n\t\t\t\t\"projects_networks\",\n\t\t\t\t\"projects_networks_restricted_uplinks\",\n\t\t\t\t\"custom_volume_backup\",\n\t\t\t\t\"backup_override_name\",\n\t\t\t\t\"storage_rsync_compression\",\n\t\t\t\t\"network_type_physical\",\n\t\t\t\t\"network_ovn_external_subnets\",\n\t\t\t\t\"network_ovn_nat\",\n\t\t\t\t\"network_ovn_external_routes_remove\",\n\t\t\t\t\"tpm_device_type\",\n\t\t\t\t\"storage_zfs_clone_copy_rebase\",\n\t\t\t\t\"gpu_mdev\",\n\t\t\t\t\"resources_pci_iommu\",\n\t\t\t\t\"resources_network_usb\",\n\t\t\t\t\"resources_disk_address\",\n\t\t\t\t\"network_physical_ovn_ingress_mode\",\n\t\t\t\t\"network_ovn_dhcp\",\n\t\t\t\t\"network_physical_routes_anycast\",\n\t\t\t\t\"projects_limits_instances\",\n\t\t\t\t\"network_state_vlan\",\n\t\t\t\t\"instance_nic_bridged_port_isolation\",\n\t\t\t\t\"instance_bulk_state_change\",\n\t\t\t\t\"network_gvrp\",\n\t\t\t\t\"instance_pool_move\",\n\t\t\t\t\"gpu_sriov\",\n\t\t\t\t\"pci_device_type\",\n\t\t\t\t\"storage_volume_state\",\n\t\t\t\t\"network_acl\",\n\t\t\t\t\"migration_stateful\",\n\t\t\t\t\"disk_state_quota\",\n\t\t\t\t\"storage_ceph_features\",\n\t\t\t\t\"projects_compression\",\n\t\t\t\t\"projects_images_remote_cache_expiry\",\n\t\t\t\t\"certificate_project\",\n\t\t\t\t\"network_ovn_acl\",\n\t\t\t\t\"projects_images_auto_update\",\n\t\t\t\t\"projects_restricted_cluster_target\",\n\t\t\t\t\"images_default_architecture\",\n\t\t\t\t\"network_ovn_acl_defaults\",\n\t\t\t\t\"gpu_mig\",\n\t\t\t\t\"project_usage\",\n\t\t\t\t\"network_bridge_acl\",\n\t\t\t\t\"warnings\",\n\t\t\t\t\"projects_restricted_backups_and_snapshots\",\n\t\t\t\t\"clustering_join_token\",\n\t\t\t\t\"clustering_description\",\n\t\t\t\t\"server_trusted_proxy\",\n\t\t\t\t\"clustering_update_cert\",\n\t\t\t\t\"storage_api_project\",\n\t\t\t\t\"server_instance_driver_operational\",\n\t\t\t\t\"server_supported_storage_drivers\",\n\t\t\t\t\"event_lifecycle_requestor_address\",\n\t\t\t\t\"resources_gpu_usb\",\n\t\t\t\t\"clustering_evacuation\",\n\t\t\t\t\"network_ovn_nat_address\",\n\t\t\t\t\"network_bgp\",\n\t\t\t\t\"network_forward\",\n\t\t\t\t\"custom_volume_refresh\",\n\t\t\t\t\"network_counters_errors_dropped\",\n\t\t\t\t\"metrics\",\n\t\t\t\t\"image_source_project\",\n\t\t\t\t\"clustering_config\",\n\t\t\t\t\"network_peer\",\n\t\t\t\t\"linux_sysctl\",\n\t\t\t\t\"network_dns\",\n\t\t\t\t\"ovn_nic_acceleration\",\n\t\t\t\t\"certificate_self_renewal\",\n\t\t\t\t\"instance_project_move\",\n\t\t\t\t\"storage_volume_project_move\",\n\t\t\t\t\"cloud_init\",\n\t\t\t\t\"network_dns_nat\",\n\t\t\t\t\"database_leader\",\n\t\t\t\t\"instance_all_projects\",\n\t\t\t\t\"clustering_groups\",\n\t\t\t\t\"ceph_rbd_du\",\n\t\t\t\t\"instance_get_full\",\n\t\t\t\t\"qemu_metrics\",\n\t\t\t\t\"gpu_mig_uuid\",\n\t\t\t\t\"event_project\",\n\t\t\t\t\"clustering_evacuation_live\",\n\t\t\t\t\"instance_allow_inconsistent_copy\",\n\t\t\t\t\"network_state_ovn\",\n\t\t\t\t\"storage_volume_api_filtering\",\n\t\t\t\t\"image_restrictions\",\n\t\t\t\t\"storage_zfs_export\",\n\t\t\t\t\"network_dns_records\",\n\t\t\t\t\"storage_zfs_reserve_space\",\n\t\t\t\t\"network_acl_log\",\n\t\t\t\t\"storage_zfs_blocksize\",\n\t\t\t\t\"metrics_cpu_seconds\",\n\t\t\t\t\"instance_snapshot_never\",\n\t\t\t\t\"certificate_token\",\n\t\t\t\t\"instance_nic_routed_neighbor_probe\",\n\t\t\t\t\"event_hub\",\n\t\t\t\t\"agent_nic_config\",\n\t\t\t\t\"projects_restricted_intercept\",\n\t\t\t\t\"metrics_authentication\",\n\t\t\t\t\"images_target_project\",\n\t\t\t\t\"images_all_projects\",\n\t\t\t\t\"cluster_migration_inconsistent_copy\",\n\t\t\t\t\"cluster_ovn_chassis\",\n\t\t\t\t\"container_syscall_intercept_sched_setscheduler\",\n\t\t\t\t\"storage_lvm_thinpool_metadata_size\",\n\t\t\t\t\"storage_volume_state_total\",\n\t\t\t\t\"instance_file_head\",\n\t\t\t\t\"instances_nic_host_name\",\n\t\t\t\t\"image_copy_profile\",\n\t\t\t\t\"container_syscall_intercept_sysinfo\",\n\t\t\t\t\"clustering_evacuation_mode\",\n\t\t\t\t\"resources_pci_vpd\",\n\t\t\t\t\"qemu_raw_conf\",\n\t\t\t\t\"storage_cephfs_fscache\",\n\t\t\t\t\"network_load_balancer\",\n\t\t\t\t\"vsock_api\",\n\t\t\t\t\"instance_ready_state\",\n\t\t\t\t\"network_bgp_holdtime\",\n\t\t\t\t\"storage_volumes_all_projects\",\n\t\t\t\t\"metrics_memory_oom_total\",\n\t\t\t\t\"storage_buckets\",\n\t\t\t\t\"storage_buckets_create_credentials\",\n\t\t\t\t\"metrics_cpu_effective_total\",\n\t\t\t\t\"projects_networks_restricted_access\",\n\t\t\t\t\"storage_buckets_local\",\n\t\t\t\t\"loki\",\n\t\t\t\t\"acme\",\n\t\t\t\t\"internal_metrics\",\n\t\t\t\t\"cluster_join_token_expiry\",\n\t\t\t\t\"remote_token_expiry\",\n\t\t\t\t\"init_preseed\",\n\t\t\t\t\"storage_volumes_created_at\",\n\t\t\t\t\"cpu_hotplug\",\n\t\t\t\t\"projects_networks_zones\",\n\t\t\t\t\"network_txqueuelen\",\n\t\t\t\t\"cluster_member_state\",\n\t\t\t\t\"instances_placement_scriptlet\",\n\t\t\t\t\"storage_pool_source_wipe\",\n\t\t\t\t\"zfs_block_mode\",\n\t\t\t\t\"instance_generation_id\",\n\t\t\t\t\"disk_io_cache\",\n\t\t\t\t\"amd_sev\",\n\t\t\t\t\"storage_pool_loop_resize\",\n\t\t\t\t\"migration_vm_live\",\n\t\t\t\t\"ovn_nic_nesting\",\n\t\t\t\t\"oidc\",\n\t\t\t\t\"network_ovn_l3only\",\n\t\t\t\t\"ovn_nic_acceleration_vdpa\",\n\t\t\t\t\"cluster_healing\",\n\t\t\t\t\"instances_state_total\",\n\t\t\t\t\"auth_user\",\n\t\t\t\t\"security_csm\",\n\t\t\t\t\"instances_rebuild\",\n\t\t\t\t\"numa_cpu_placement\",\n\t\t\t\t\"custom_volume_iso\",\n\t\t\t\t\"network_allocations\",\n\t\t\t\t\"zfs_delegate\",\n\t\t\t\t\"storage_api_remote_volume_snapshot_copy\",\n\t\t\t\t\"operations_get_query_all_projects\",\n\t\t\t\t\"metadata_configuration\",\n\t\t\t\t\"syslog_socket\",\n\t\t\t\t\"event_lifecycle_name_and_project\",\n\t\t\t\t\"instances_nic_limits_priority\",\n\t\t\t\t\"disk_initial_volume_configuration\",\n\t\t\t\t\"operation_wait\",\n\t\t\t\t\"image_restriction_privileged\",\n\t\t\t\t\"cluster_internal_custom_volume_copy\",\n\t\t\t\t\"disk_io_bus\",\n\t\t\t\t\"storage_cephfs_create_missing\",\n\t\t\t\t\"instance_move_config\",\n\t\t\t\t\"ovn_ssl_config\",\n\t\t\t\t\"certificate_description\",\n\t\t\t\t\"disk_io_bus_virtio_blk\",\n\t\t\t\t\"loki_config_instance\",\n\t\t\t\t\"instance_create_start\",\n\t\t\t\t\"clustering_evacuation_stop_options\",\n\t\t\t\t\"boot_host_shutdown_action\",\n\t\t\t\t\"agent_config_drive\",\n\t\t\t\t\"network_state_ovn_lr\",\n\t\t\t\t\"image_template_permissions\",\n\t\t\t\t\"storage_bucket_backup\",\n\t\t\t\t\"storage_lvm_cluster\",\n\t\t\t\t\"shared_custom_block_volumes\",\n\t\t\t\t\"auth_tls_jwt\",\n\t\t\t\t\"oidc_claim\",\n\t\t\t\t\"device_usb_serial\",\n\t\t\t\t\"numa_cpu_balanced\",\n\t\t\t\t\"image_restriction_nesting\",\n\t\t\t\t\"network_integrations\",\n\t\t\t\t\"instance_memory_swap_bytes\",\n\t\t\t\t\"network_bridge_external_create\",\n\t\t\t\t\"network_zones_all_projects\",\n\t\t\t\t\"storage_zfs_vdev\",\n\t\t\t\t\"container_migration_stateful\",\n\t\t\t\t\"profiles_all_projects\",\n\t\t\t\t\"instances_scriptlet_get_instances\",\n\t\t\t\t\"instances_scriptlet_get_cluster_members\",\n\t\t\t\t\"instances_scriptlet_get_project\",\n\t\t\t\t\"network_acl_stateless\",\n\t\t\t\t\"instance_state_started_at\",\n\t\t\t\t\"networks_all_projects\",\n\t\t\t\t\"network_acls_all_projects\",\n\t\t\t\t\"storage_buckets_all_projects\",\n\t\t\t\t\"resources_load\",\n\t\t\t\t\"instance_access\",\n\t\t\t\t\"project_access\",\n\t\t\t\t\"projects_force_delete\",\n\t\t\t\t\"resources_cpu_flags\",\n\t\t\t\t\"disk_io_bus_cache_filesystem\",\n\t\t\t\t\"instance_oci\"\n\t\t\t],\n\t\t\t\"api_status\": \"stable\",\n\t\t\t\"api_version\": \"1.0\",\n\t\t\t\"auth\": \"trusted\",\n\t\t\t\"public\": false,\n\t\t\t\"auth_methods\": [\n\t\t\t\t\"tls\"\n\t\t\t],\n\t\t\t\"auth_user_name\": \"a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63\",\n\t\t\t\"auth_user_method\": \"tls\",\n\t\t\t\"environment\": {\n\t\t\t\t\"addresses\": [\n\t\t\t\t\t\"192.168.200.240:8443\",\n\t\t\t\t\t\"[2001:a61:3b74:9b01:c023:80ff:fedf:d000]:8443\",\n\t\t\t\t\t\"[2001:a61:3b74:9b01::240]:8443\"\n\t\t\t\t],\n\t\t\t\t\"architectures\": [\n\t\t\t\t\t\"x86_64\",\n\t\t\t\t\t\"i686\"\n\t\t\t\t],\n\t\t\t\t\"certificate\": \"-----BEGIN CERTIFICATE-----\\nMIICBzCCAY2gAwIBAgIRAIuF/523EGEDUsETcU/gGbQwCgYIKoZIzj0EAwMwNDEZ\\nMBcGA1UEChMQTGludXggQ29udGFpbmVyczEXMBUGA1UEAwwOcm9vdEBzdGFyZ2F6\\nZXIwHhcNMjQwNjI5MTEyNDI5WhcNMzQwNjI3MTEyNDI5WjA0MRkwFwYDVQQKExBM\\naW51eCBDb250YWluZXJzMRcwFQYDVQQDDA5yb290QHN0YXJnYXplcjB2MBAGByqG\\nSM49AgEGBSuBBAAiA2IABLMZW1hqNseQRxYhE27v3yiYXHyYwvnyce5eRiGHulZv\\nVbJFkZiwW8wLqB9mnp0RAJffNdFX/WJzBP/kdyK2pCx7SKdwbw44hPjy5O8HBKwh\\nMIvCkTxjeJhq74twfTdf86NjMGEwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoG\\nCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwLAYDVR0RBCUwI4IJc3RhcmdhemVyhwR/\\nAAABhxAAAAAAAAAAAAAAAAAAAAABMAoGCCqGSM49BAMDA2gAMGUCMQC6hCWcuB35\\nUPddIROXPirw/zJIkvA1RjuPrg6FtbPO8bpLUjkzgXlm8SBNZC7FPhYCMGgfVegM\\nXgl9TKno8PjjhThWN3pm4ABS2LFQJyhwXXipZYHIRn1Z0TqYtmQo7qdBnA==\\n-----END CERTIFICATE-----\\n\",\n\t\t\t\t\"certificate_fingerprint\": \"6b64d8551f31655d986c790b8c4f724b9968d585c1b5649f1c8d02f748bf99bc\",\n\t\t\t\t\"driver\": \"lxc\",\n\t\t\t\t\"driver_version\": \"6.0.1\",\n\t\t\t\t\"firewall\": \"nftables\",\n\t\t\t\t\"kernel\": \"Linux\",\n\t\t\t\t\"kernel_architecture\": \"x86_64\",\n\t\t\t\t\"kernel_features\": {\n\t\t\t\t\t\"idmapped_mounts\": \"true\",\n\t\t\t\t\t\"netnsid_getifaddrs\": \"true\",\n\t\t\t\t\t\"seccomp_listener\": \"true\",\n\t\t\t\t\t\"seccomp_listener_continue\": \"true\",\n\t\t\t\t\t\"uevent_injection\": \"true\",\n\t\t\t\t\t\"unpriv_binfmt\": \"true\",\n\t\t\t\t\t\"unpriv_fscaps\": \"true\"\n\t\t\t\t},\n\t\t\t\t\"kernel_version\": \"6.9.10-arch1-1\",\n\t\t\t\t\"lxc_features\": {\n\t\t\t\t\t\"cgroup2\": \"true\",\n\t\t\t\t\t\"core_scheduling\": \"true\",\n\t\t\t\t\t\"devpts_fd\": \"true\",\n\t\t\t\t\t\"idmapped_mounts_v2\": \"true\",\n\t\t\t\t\t\"mount_injection_file\": \"true\",\n\t\t\t\t\t\"network_gateway_device_route\": \"true\",\n\t\t\t\t\t\"network_ipvlan\": \"true\",\n\t\t\t\t\t\"network_l2proxy\": \"true\",\n\t\t\t\t\t\"network_phys_macvlan_mtu\": \"true\",\n\t\t\t\t\t\"network_veth_router\": \"true\",\n\t\t\t\t\t\"pidfd\": \"true\",\n\t\t\t\t\t\"seccomp_allow_deny_syntax\": \"true\",\n\t\t\t\t\t\"seccomp_notify\": \"true\",\n\t\t\t\t\t\"seccomp_proxy_send_notify_fd\": \"true\"\n\t\t\t\t},\n\t\t\t\t\"os_name\": \"Arch Linux\",\n\t\t\t\t\"os_version\": \"\",\n\t\t\t\t\"project\": \"default\",\n\t\t\t\t\"server\": \"incus\",\n\t\t\t\t\"server_clustered\": false,\n\t\t\t\t\"server_event_mode\": \"full-mesh\",\n\t\t\t\t\"server_name\": \"stargazer\",\n\t\t\t\t\"server_pid\": 1083,\n\t\t\t\t\"server_version\": \"6.3\",\n\t\t\t\t\"storage\": \"btrfs\",\n\t\t\t\t\"storage_version\": \"6.9.2\",\n\t\t\t\t\"storage_supported_drivers\": [\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"btrfs\",\n\t\t\t\t\t\t\"Version\": \"6.9.2\",\n\t\t\t\t\t\t\"Remote\": false\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"dir\",\n\t\t\t\t\t\t\"Version\": \"1\",\n\t\t\t\t\t\t\"Remote\": false\n\t\t\t\t\t}\n\t\t\t\t]\n\t\t\t}\n\t\t}\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {\n\t\t\t\"config\": {\n\t\t\t\t\"features.images\": \"true\",\n\t\t\t\t\"features.networks\": \"true\",\n\t\t\t\t\"features.networks.zones\": \"true\",\n\t\t\t\t\"features.profiles\": \"true\",\n\t\t\t\t\"features.storage.buckets\": \"true\",\n\t\t\t\t\"features.storage.volumes\": \"true\"\n\t\t\t},\n\t\t\t\"description\": \"Default Incus project\",\n\t\t\t\"name\": \"default\",\n\t\t\t\"used_by\": [\n\t\t\t\t\"/1.0/profiles/default\",\n\t\t\t\t\"/1.0/profiles/k3s\"\n\t\t\t]\n\t\t}\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url="/1.0/projects?recursion=1" username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url=/1.0 username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url="/1.0/operations?all-projects=true&recursion=1" username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url="/1.0/certificates?recursion=1" username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {}\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {\n\t\t\t\"config\": {\n\t\t\t\t\"core.https_address\": \":8443\",\n\t\t\t\t\"core.storage_buckets_address\": \":8555\"\n\t\t\t},\n\t\t\t\"api_extensions\": [\n\t\t\t\t\"storage_zfs_remove_snapshots\",\n\t\t\t\t\"container_host_shutdown_timeout\",\n\t\t\t\t\"container_stop_priority\",\n\t\t\t\t\"container_syscall_filtering\",\n\t\t\t\t\"auth_pki\",\n\t\t\t\t\"container_last_used_at\",\n\t\t\t\t\"etag\",\n\t\t\t\t\"patch\",\n\t\t\t\t\"usb_devices\",\n\t\t\t\t\"https_allowed_credentials\",\n\t\t\t\t\"image_compression_algorithm\",\n\t\t\t\t\"directory_manipulation\",\n\t\t\t\t\"container_cpu_time\",\n\t\t\t\t\"storage_zfs_use_refquota\",\n\t\t\t\t\"storage_lvm_mount_options\",\n\t\t\t\t\"network\",\n\t\t\t\t\"profile_usedby\",\n\t\t\t\t\"container_push\",\n\t\t\t\t\"container_exec_recording\",\n\t\t\t\t\"certificate_update\",\n\t\t\t\t\"container_exec_signal_handling\",\n\t\t\t\t\"gpu_devices\",\n\t\t\t\t\"container_image_properties\",\n\t\t\t\t\"migration_progress\",\n\t\t\t\t\"id_map\",\n\t\t\t\t\"network_firewall_filtering\",\n\t\t\t\t\"network_routes\",\n\t\t\t\t\"storage\",\n\t\t\t\t\"file_delete\",\n\t\t\t\t\"file_append\",\n\t\t\t\t\"network_dhcp_expiry\",\n\t\t\t\t\"storage_lvm_vg_rename\",\n\t\t\t\t\"storage_lvm_thinpool_rename\",\n\t\t\t\t\"network_vlan\",\n\t\t\t\t\"image_create_aliases\",\n\t\t\t\t\"container_stateless_copy\",\n\t\t\t\t\"container_only_migration\",\n\t\t\t\t\"storage_zfs_clone_copy\",\n\t\t\t\t\"unix_device_rename\",\n\t\t\t\t\"storage_lvm_use_thinpool\",\n\t\t\t\t\"storage_rsync_bwlimit\",\n\t\t\t\t\"network_vxlan_interface\",\n\t\t\t\t\"storage_btrfs_mount_options\",\n\t\t\t\t\"entity_description\",\n\t\t\t\t\"image_force_refresh\",\n\t\t\t\t\"storage_lvm_lv_resizing\",\n\t\t\t\t\"id_map_base\",\n\t\t\t\t\"file_symlinks\",\n\t\t\t\t\"container_push_target\",\n\t\t\t\t\"network_vlan_physical\",\n\t\t\t\t\"storage_images_delete\",\n\t\t\t\t\"container_edit_metadata\",\n\t\t\t\t\"container_snapshot_stateful_migration\",\n\t\t\t\t\"storage_driver_ceph\",\n\t\t\t\t\"storage_ceph_user_name\",\n\t\t\t\t\"resource_limits\",\n\t\t\t\t\"storage_volatile_initial_source\",\n\t\t\t\t\"storage_ceph_force_osd_reuse\",\n\t\t\t\t\"storage_block_filesystem_btrfs\",\n\t\t\t\t\"resources\",\n\t\t\t\t\"kernel_limits\",\n\t\t\t\t\"storage_api_volume_rename\",\n\t\t\t\t\"network_sriov\",\n\t\t\t\t\"console\",\n\t\t\t\t\"restrict_dev_incus\",\n\t\t\t\t\"migration_pre_copy\",\n\t\t\t\t\"infiniband\",\n\t\t\t\t\"dev_incus_events\",\n\t\t\t\t\"proxy\",\n\t\t\t\t\"network_dhcp_gateway\",\n\t\t\t\t\"file_get_symlink\",\n\t\t\t\t\"network_leases\",\n\t\t\t\t\"unix_device_hotplug\",\n\t\t\t\t\"storage_api_local_volume_handling\",\n\t\t\t\t\"operation_description\",\n\t\t\t\t\"clustering\",\n\t\t\t\t\"event_lifecycle\",\n\t\t\t\t\"storage_api_remote_volume_handling\",\n\t\t\t\t\"nvidia_runtime\",\n\t\t\t\t\"container_mount_propagation\",\n\t\t\t\t\"container_backup\",\n\t\t\t\t\"dev_incus_images\",\n\t\t\t\t\"container_local_cross_pool_handling\",\n\t\t\t\t\"proxy_unix\",\n\t\t\t\t\"proxy_udp\",\n\t\t\t\t\"clustering_join\",\n\t\t\t\t\"proxy_tcp_udp_multi_port_handling\",\n\t\t\t\t\"network_state\",\n\t\t\t\t\"proxy_unix_dac_properties\",\n\t\t\t\t\"container_protection_delete\",\n\t\t\t\t\"unix_priv_drop\",\n\t\t\t\t\"pprof_http\",\n\t\t\t\t\"proxy_haproxy_protocol\",\n\t\t\t\t\"network_hwaddr\",\n\t\t\t\t\"proxy_nat\",\n\t\t\t\t\"network_nat_order\",\n\t\t\t\t\"container_full\",\n\t\t\t\t\"backup_compression\",\n\t\t\t\t\"nvidia_runtime_config\",\n\t\t\t\t\"storage_api_volume_snapshots\",\n\t\t\t\t\"storage_unmapped\",\n\t\t\t\t\"projects\",\n\t\t\t\t\"network_vxlan_ttl\",\n\t\t\t\t\"container_incremental_copy\",\n\t\t\t\t\"usb_optional_vendorid\",\n\t\t\t\t\"snapshot_scheduling\",\n\t\t\t\t\"snapshot_schedule_aliases\",\n\t\t\t\t\"container_copy_project\",\n\t\t\t\t\"clustering_server_address\",\n\t\t\t\t\"clustering_image_replication\",\n\t\t\t\t\"container_protection_shift\",\n\t\t\t\t\"snapshot_expiry\",\n\t\t\t\t\"container_backup_override_pool\",\n\t\t\t\t\"snapshot_expiry_creation\",\n\t\t\t\t\"network_leases_location\",\n\t\t\t\t\"resources_cpu_socket\",\n\t\t\t\t\"resources_gpu\",\n\t\t\t\t\"resources_numa\",\n\t\t\t\t\"kernel_features\",\n\t\t\t\t\"id_map_current\",\n\t\t\t\t\"event_location\",\n\t\t\t\t\"storage_api_remote_volume_snapshots\",\n\t\t\t\t\"network_nat_address\",\n\t\t\t\t\"container_nic_routes\",\n\t\t\t\t\"cluster_internal_copy\",\n\t\t\t\t\"seccomp_notify\",\n\t\t\t\t\"lxc_features\",\n\t\t\t\t\"container_nic_ipvlan\",\n\t\t\t\t\"network_vlan_sriov\",\n\t\t\t\t\"storage_cephfs\",\n\t\t\t\t\"container_nic_ipfilter\",\n\t\t\t\t\"resources_v2\",\n\t\t\t\t\"container_exec_user_group_cwd\",\n\t\t\t\t\"container_syscall_intercept\",\n\t\t\t\t\"container_disk_shift\",\n\t\t\t\t\"storage_shifted\",\n\t\t\t\t\"resources_infiniband\",\n\t\t\t\t\"daemon_storage\",\n\t\t\t\t\"instances\",\n\t\t\t\t\"image_types\",\n\t\t\t\t\"resources_disk_sata\",\n\t\t\t\t\"clustering_roles\",\n\t\t\t\t\"images_expiry\",\n\t\t\t\t\"resources_network_firmware\",\n\t\t\t\t\"backup_compression_algorithm\",\n\t\t\t\t\"ceph_data_pool_name\",\n\t\t\t\t\"container_syscall_intercept_mount\",\n\t\t\t\t\"compression_squashfs\",\n\t\t\t\t\"container_raw_mount\",\n\t\t\t\t\"container_nic_routed\",\n\t\t\t\t\"container_syscall_intercept_mount_fuse\",\n\t\t\t\t\"container_disk_ceph\",\n\t\t\t\t\"virtual-machines\",\n\t\t\t\t\"image_profiles\",\n\t\t\t\t\"clustering_architecture\",\n\t\t\t\t\"resources_disk_id\",\n\t\t\t\t\"storage_lvm_stripes\",\n\t\t\t\t\"vm_boot_priority\",\n\t\t\t\t\"unix_hotplug_devices\",\n\t\t\t\t\"api_filtering\",\n\t\t\t\t\"instance_nic_network\",\n\t\t\t\t\"clustering_sizing\",\n\t\t\t\t\"firewall_driver\",\n\t\t\t\t\"projects_limits\",\n\t\t\t\t\"container_syscall_intercept_hugetlbfs\",\n\t\t\t\t\"limits_hugepages\",\n\t\t\t\t\"container_nic_routed_gateway\",\n\t\t\t\t\"projects_restrictions\",\n\t\t\t\t\"custom_volume_snapshot_expiry\",\n\t\t\t\t\"volume_snapshot_scheduling\",\n\t\t\t\t\"trust_ca_certificates\",\n\t\t\t\t\"snapshot_disk_usage\",\n\t\t\t\t\"clustering_edit_roles\",\n\t\t\t\t\"container_nic_routed_host_address\",\n\t\t\t\t\"container_nic_ipvlan_gateway\",\n\t\t\t\t\"resources_usb_pci\",\n\t\t\t\t\"resources_cpu_threads_numa\",\n\t\t\t\t\"resources_cpu_core_die\",\n\t\t\t\t\"api_os\",\n\t\t\t\t\"container_nic_routed_host_table\",\n\t\t\t\t\"container_nic_ipvlan_host_table\",\n\t\t\t\t\"container_nic_ipvlan_mode\",\n\t\t\t\t\"resources_system\",\n\t\t\t\t\"images_push_relay\",\n\t\t\t\t\"network_dns_search\",\n\t\t\t\t\"container_nic_routed_limits\",\n\t\t\t\t\"instance_nic_bridged_vlan\",\n\t\t\t\t\"network_state_bond_bridge\",\n\t\t\t\t\"usedby_consistency\",\n\t\t\t\t\"custom_block_volumes\",\n\t\t\t\t\"clustering_failure_domains\",\n\t\t\t\t\"resources_gpu_mdev\",\n\t\t\t\t\"console_vga_type\",\n\t\t\t\t\"projects_limits_disk\",\n\t\t\t\t\"network_type_macvlan\",\n\t\t\t\t\"network_type_sriov\",\n\t\t\t\t\"container_syscall_intercept_bpf_devices\",\n\t\t\t\t\"network_type_ovn\",\n\t\t\t\t\"projects_networks\",\n\t\t\t\t\"projects_networks_restricted_uplinks\",\n\t\t\t\t\"custom_volume_backup\",\n\t\t\t\t\"backup_override_name\",\n\t\t\t\t\"storage_rsync_compression\",\n\t\t\t\t\"network_type_physical\",\n\t\t\t\t\"network_ovn_external_subnets\",\n\t\t\t\t\"network_ovn_nat\",\n\t\t\t\t\"network_ovn_external_routes_remove\",\n\t\t\t\t\"tpm_device_type\",\n\t\t\t\t\"storage_zfs_clone_copy_rebase\",\n\t\t\t\t\"gpu_mdev\",\n\t\t\t\t\"resources_pci_iommu\",\n\t\t\t\t\"resources_network_usb\",\n\t\t\t\t\"resources_disk_address\",\n\t\t\t\t\"network_physical_ovn_ingress_mode\",\n\t\t\t\t\"network_ovn_dhcp\",\n\t\t\t\t\"network_physical_routes_anycast\",\n\t\t\t\t\"projects_limits_instances\",\n\t\t\t\t\"network_state_vlan\",\n\t\t\t\t\"instance_nic_bridged_port_isolation\",\n\t\t\t\t\"instance_bulk_state_change\",\n\t\t\t\t\"network_gvrp\",\n\t\t\t\t\"instance_pool_move\",\n\t\t\t\t\"gpu_sriov\",\n\t\t\t\t\"pci_device_type\",\n\t\t\t\t\"storage_volume_state\",\n\t\t\t\t\"network_acl\",\n\t\t\t\t\"migration_stateful\",\n\t\t\t\t\"disk_state_quota\",\n\t\t\t\t\"storage_ceph_features\",\n\t\t\t\t\"projects_compression\",\n\t\t\t\t\"projects_images_remote_cache_expiry\",\n\t\t\t\t\"certificate_project\",\n\t\t\t\t\"network_ovn_acl\",\n\t\t\t\t\"projects_images_auto_update\",\n\t\t\t\t\"projects_restricted_cluster_target\",\n\t\t\t\t\"images_default_architecture\",\n\t\t\t\t\"network_ovn_acl_defaults\",\n\t\t\t\t\"gpu_mig\",\n\t\t\t\t\"project_usage\",\n\t\t\t\t\"network_bridge_acl\",\n\t\t\t\t\"warnings\",\n\t\t\t\t\"projects_restricted_backups_and_snapshots\",\n\t\t\t\t\"clustering_join_token\",\n\t\t\t\t\"clustering_description\",\n\t\t\t\t\"server_trusted_proxy\",\n\t\t\t\t\"clustering_update_cert\",\n\t\t\t\t\"storage_api_project\",\n\t\t\t\t\"server_instance_driver_operational\",\n\t\t\t\t\"server_supported_storage_drivers\",\n\t\t\t\t\"event_lifecycle_requestor_address\",\n\t\t\t\t\"resources_gpu_usb\",\n\t\t\t\t\"clustering_evacuation\",\n\t\t\t\t\"network_ovn_nat_address\",\n\t\t\t\t\"network_bgp\",\n\t\t\t\t\"network_forward\",\n\t\t\t\t\"custom_volume_refresh\",\n\t\t\t\t\"network_counters_errors_dropped\",\n\t\t\t\t\"metrics\",\n\t\t\t\t\"image_source_project\",\n\t\t\t\t\"clustering_config\",\n\t\t\t\t\"network_peer\",\n\t\t\t\t\"linux_sysctl\",\n\t\t\t\t\"network_dns\",\n\t\t\t\t\"ovn_nic_acceleration\",\n\t\t\t\t\"certificate_self_renewal\",\n\t\t\t\t\"instance_project_move\",\n\t\t\t\t\"storage_volume_project_move\",\n\t\t\t\t\"cloud_init\",\n\t\t\t\t\"network_dns_nat\",\n\t\t\t\t\"database_leader\",\n\t\t\t\t\"instance_all_projects\",\n\t\t\t\t\"clustering_groups\",\n\t\t\t\t\"ceph_rbd_du\",\n\t\t\t\t\"instance_get_full\",\n\t\t\t\t\"qemu_metrics\",\n\t\t\t\t\"gpu_mig_uuid\",\n\t\t\t\t\"event_project\",\n\t\t\t\t\"clustering_evacuation_live\",\n\t\t\t\t\"instance_allow_inconsistent_copy\",\n\t\t\t\t\"network_state_ovn\",\n\t\t\t\t\"storage_volume_api_filtering\",\n\t\t\t\t\"image_restrictions\",\n\t\t\t\t\"storage_zfs_export\",\n\t\t\t\t\"network_dns_records\",\n\t\t\t\t\"storage_zfs_reserve_space\",\n\t\t\t\t\"network_acl_log\",\n\t\t\t\t\"storage_zfs_blocksize\",\n\t\t\t\t\"metrics_cpu_seconds\",\n\t\t\t\t\"instance_snapshot_never\",\n\t\t\t\t\"certificate_token\",\n\t\t\t\t\"instance_nic_routed_neighbor_probe\",\n\t\t\t\t\"event_hub\",\n\t\t\t\t\"agent_nic_config\",\n\t\t\t\t\"projects_restricted_intercept\",\n\t\t\t\t\"metrics_authentication\",\n\t\t\t\t\"images_target_project\",\n\t\t\t\t\"images_all_projects\",\n\t\t\t\t\"cluster_migration_inconsistent_copy\",\n\t\t\t\t\"cluster_ovn_chassis\",\n\t\t\t\t\"container_syscall_intercept_sched_setscheduler\",\n\t\t\t\t\"storage_lvm_thinpool_metadata_size\",\n\t\t\t\t\"storage_volume_state_total\",\n\t\t\t\t\"instance_file_head\",\n\t\t\t\t\"instances_nic_host_name\",\n\t\t\t\t\"image_copy_profile\",\n\t\t\t\t\"container_syscall_intercept_sysinfo\",\n\t\t\t\t\"clustering_evacuation_mode\",\n\t\t\t\t\"resources_pci_vpd\",\n\t\t\t\t\"qemu_raw_conf\",\n\t\t\t\t\"storage_cephfs_fscache\",\n\t\t\t\t\"network_load_balancer\",\n\t\t\t\t\"vsock_api\",\n\t\t\t\t\"instance_ready_state\",\n\t\t\t\t\"network_bgp_holdtime\",\n\t\t\t\t\"storage_volumes_all_projects\",\n\t\t\t\t\"metrics_memory_oom_total\",\n\t\t\t\t\"storage_buckets\",\n\t\t\t\t\"storage_buckets_create_credentials\",\n\t\t\t\t\"metrics_cpu_effective_total\",\n\t\t\t\t\"projects_networks_restricted_access\",\n\t\t\t\t\"storage_buckets_local\",\n\t\t\t\t\"loki\",\n\t\t\t\t\"acme\",\n\t\t\t\t\"internal_metrics\",\n\t\t\t\t\"cluster_join_token_expiry\",\n\t\t\t\t\"remote_token_expiry\",\n\t\t\t\t\"init_preseed\",\n\t\t\t\t\"storage_volumes_created_at\",\n\t\t\t\t\"cpu_hotplug\",\n\t\t\t\t\"projects_networks_zones\",\n\t\t\t\t\"network_txqueuelen\",\n\t\t\t\t\"cluster_member_state\",\n\t\t\t\t\"instances_placement_scriptlet\",\n\t\t\t\t\"storage_pool_source_wipe\",\n\t\t\t\t\"zfs_block_mode\",\n\t\t\t\t\"instance_generation_id\",\n\t\t\t\t\"disk_io_cache\",\n\t\t\t\t\"amd_sev\",\n\t\t\t\t\"storage_pool_loop_resize\",\n\t\t\t\t\"migration_vm_live\",\n\t\t\t\t\"ovn_nic_nesting\",\n\t\t\t\t\"oidc\",\n\t\t\t\t\"network_ovn_l3only\",\n\t\t\t\t\"ovn_nic_acceleration_vdpa\",\n\t\t\t\t\"cluster_healing\",\n\t\t\t\t\"instances_state_total\",\n\t\t\t\t\"auth_user\",\n\t\t\t\t\"security_csm\",\n\t\t\t\t\"instances_rebuild\",\n\t\t\t\t\"numa_cpu_placement\",\n\t\t\t\t\"custom_volume_iso\",\n\t\t\t\t\"network_allocations\",\n\t\t\t\t\"zfs_delegate\",\n\t\t\t\t\"storage_api_remote_volume_snapshot_copy\",\n\t\t\t\t\"operations_get_query_all_projects\",\n\t\t\t\t\"metadata_configuration\",\n\t\t\t\t\"syslog_socket\",\n\t\t\t\t\"event_lifecycle_name_and_project\",\n\t\t\t\t\"instances_nic_limits_priority\",\n\t\t\t\t\"disk_initial_volume_configuration\",\n\t\t\t\t\"operation_wait\",\n\t\t\t\t\"image_restriction_privileged\",\n\t\t\t\t\"cluster_internal_custom_volume_copy\",\n\t\t\t\t\"disk_io_bus\",\n\t\t\t\t\"storage_cephfs_create_missing\",\n\t\t\t\t\"instance_move_config\",\n\t\t\t\t\"ovn_ssl_config\",\n\t\t\t\t\"certificate_description\",\n\t\t\t\t\"disk_io_bus_virtio_blk\",\n\t\t\t\t\"loki_config_instance\",\n\t\t\t\t\"instance_create_start\",\n\t\t\t\t\"clustering_evacuation_stop_options\",\n\t\t\t\t\"boot_host_shutdown_action\",\n\t\t\t\t\"agent_config_drive\",\n\t\t\t\t\"network_state_ovn_lr\",\n\t\t\t\t\"image_template_permissions\",\n\t\t\t\t\"storage_bucket_backup\",\n\t\t\t\t\"storage_lvm_cluster\",\n\t\t\t\t\"shared_custom_block_volumes\",\n\t\t\t\t\"auth_tls_jwt\",\n\t\t\t\t\"oidc_claim\",\n\t\t\t\t\"device_usb_serial\",\n\t\t\t\t\"numa_cpu_balanced\",\n\t\t\t\t\"image_restriction_nesting\",\n\t\t\t\t\"network_integrations\",\n\t\t\t\t\"instance_memory_swap_bytes\",\n\t\t\t\t\"network_bridge_external_create\",\n\t\t\t\t\"network_zones_all_projects\",\n\t\t\t\t\"storage_zfs_vdev\",\n\t\t\t\t\"container_migration_stateful\",\n\t\t\t\t\"profiles_all_projects\",\n\t\t\t\t\"instances_scriptlet_get_instances\",\n\t\t\t\t\"instances_scriptlet_get_cluster_members\",\n\t\t\t\t\"instances_scriptlet_get_project\",\n\t\t\t\t\"network_acl_stateless\",\n\t\t\t\t\"instance_state_started_at\",\n\t\t\t\t\"networks_all_projects\",\n\t\t\t\t\"network_acls_all_projects\",\n\t\t\t\t\"storage_buckets_all_projects\",\n\t\t\t\t\"resources_load\",\n\t\t\t\t\"instance_access\",\n\t\t\t\t\"project_access\",\n\t\t\t\t\"projects_force_delete\",\n\t\t\t\t\"resources_cpu_flags\",\n\t\t\t\t\"disk_io_bus_cache_filesystem\",\n\t\t\t\t\"instance_oci\"\n\t\t\t],\n\t\t\t\"api_status\": \"stable\",\n\t\t\t\"api_version\": \"1.0\",\n\t\t\t\"auth\": \"trusted\",\n\t\t\t\"public\": false,\n\t\t\t\"auth_methods\": [\n\t\t\t\t\"tls\"\n\t\t\t],\n\t\t\t\"auth_user_name\": \"a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63\",\n\t\t\t\"auth_user_method\": \"tls\",\n\t\t\t\"environment\": {\n\t\t\t\t\"addresses\": [\n\t\t\t\t\t\"192.168.200.240:8443\",\n\t\t\t\t\t\"[2001:a61:3b74:9b01:c023:80ff:fedf:d000]:8443\",\n\t\t\t\t\t\"[2001:a61:3b74:9b01::240]:8443\"\n\t\t\t\t],\n\t\t\t\t\"architectures\": [\n\t\t\t\t\t\"x86_64\",\n\t\t\t\t\t\"i686\"\n\t\t\t\t],\n\t\t\t\t\"certificate\": \"-----BEGIN CERTIFICATE-----\\nMIICBzCCAY2gAwIBAgIRAIuF/523EGEDUsETcU/gGbQwCgYIKoZIzj0EAwMwNDEZ\\nMBcGA1UEChMQTGludXggQ29udGFpbmVyczEXMBUGA1UEAwwOcm9vdEBzdGFyZ2F6\\nZXIwHhcNMjQwNjI5MTEyNDI5WhcNMzQwNjI3MTEyNDI5WjA0MRkwFwYDVQQKExBM\\naW51eCBDb250YWluZXJzMRcwFQYDVQQDDA5yb290QHN0YXJnYXplcjB2MBAGByqG\\nSM49AgEGBSuBBAAiA2IABLMZW1hqNseQRxYhE27v3yiYXHyYwvnyce5eRiGHulZv\\nVbJFkZiwW8wLqB9mnp0RAJffNdFX/WJzBP/kdyK2pCx7SKdwbw44hPjy5O8HBKwh\\nMIvCkTxjeJhq74twfTdf86NjMGEwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoG\\nCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwLAYDVR0RBCUwI4IJc3RhcmdhemVyhwR/\\nAAABhxAAAAAAAAAAAAAAAAAAAAABMAoGCCqGSM49BAMDA2gAMGUCMQC6hCWcuB35\\nUPddIROXPirw/zJIkvA1RjuPrg6FtbPO8bpLUjkzgXlm8SBNZC7FPhYCMGgfVegM\\nXgl9TKno8PjjhThWN3pm4ABS2LFQJyhwXXipZYHIRn1Z0TqYtmQo7qdBnA==\\n-----END CERTIFICATE-----\\n\",\n\t\t\t\t\"certificate_fingerprint\": \"6b64d8551f31655d986c790b8c4f724b9968d585c1b5649f1c8d02f748bf99bc\",\n\t\t\t\t\"driver\": \"lxc\",\n\t\t\t\t\"driver_version\": \"6.0.1\",\n\t\t\t\t\"firewall\": \"nftables\",\n\t\t\t\t\"kernel\": \"Linux\",\n\t\t\t\t\"kernel_architecture\": \"x86_64\",\n\t\t\t\t\"kernel_features\": {\n\t\t\t\t\t\"idmapped_mounts\": \"true\",\n\t\t\t\t\t\"netnsid_getifaddrs\": \"true\",\n\t\t\t\t\t\"seccomp_listener\": \"true\",\n\t\t\t\t\t\"seccomp_listener_continue\": \"true\",\n\t\t\t\t\t\"uevent_injection\": \"true\",\n\t\t\t\t\t\"unpriv_binfmt\": \"true\",\n\t\t\t\t\t\"unpriv_fscaps\": \"true\"\n\t\t\t\t},\n\t\t\t\t\"kernel_version\": \"6.9.10-arch1-1\",\n\t\t\t\t\"lxc_features\": {\n\t\t\t\t\t\"cgroup2\": \"true\",\n\t\t\t\t\t\"core_scheduling\": \"true\",\n\t\t\t\t\t\"devpts_fd\": \"true\",\n\t\t\t\t\t\"idmapped_mounts_v2\": \"true\",\n\t\t\t\t\t\"mount_injection_file\": \"true\",\n\t\t\t\t\t\"network_gateway_device_route\": \"true\",\n\t\t\t\t\t\"network_ipvlan\": \"true\",\n\t\t\t\t\t\"network_l2proxy\": \"true\",\n\t\t\t\t\t\"network_phys_macvlan_mtu\": \"true\",\n\t\t\t\t\t\"network_veth_router\": \"true\",\n\t\t\t\t\t\"pidfd\": \"true\",\n\t\t\t\t\t\"seccomp_allow_deny_syntax\": \"true\",\n\t\t\t\t\t\"seccomp_notify\": \"true\",\n\t\t\t\t\t\"seccomp_proxy_send_notify_fd\": \"true\"\n\t\t\t\t},\n\t\t\t\t\"os_name\": \"Arch Linux\",\n\t\t\t\t\"os_version\": \"\",\n\t\t\t\t\"project\": \"default\",\n\t\t\t\t\"server\": \"incus\",\n\t\t\t\t\"server_clustered\": false,\n\t\t\t\t\"server_event_mode\": \"full-mesh\",\n\t\t\t\t\"server_name\": \"stargazer\",\n\t\t\t\t\"server_pid\": 1083,\n\t\t\t\t\"server_version\": \"6.3\",\n\t\t\t\t\"storage\": \"btrfs\",\n\t\t\t\t\"storage_version\": \"6.9.2\",\n\t\t\t\t\"storage_supported_drivers\": [\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"btrfs\",\n\t\t\t\t\t\t\"Version\": \"6.9.2\",\n\t\t\t\t\t\t\"Remote\": false\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"dir\",\n\t\t\t\t\t\t\"Version\": \"1\",\n\t\t\t\t\t\t\"Remote\": false\n\t\t\t\t\t}\n\t\t\t\t]\n\t\t\t}\n\t\t}\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": [\n\t\t\t{\n\t\t\t\t\"config\": {\n\t\t\t\t\t\"features.images\": \"true\",\n\t\t\t\t\t\"features.networks\": \"true\",\n\t\t\t\t\t\"features.networks.zones\": \"true\",\n\t\t\t\t\t\"features.profiles\": \"true\",\n\t\t\t\t\t\"features.storage.buckets\": \"true\",\n\t\t\t\t\t\"features.storage.volumes\": \"true\"\n\t\t\t\t},\n\t\t\t\t\"description\": \"Default Incus project\",\n\t\t\t\t\"name\": \"default\",\n\t\t\t\t\"used_by\": [\n\t\t\t\t\t\"/1.0/profiles/default\",\n\t\t\t\t\t\"/1.0/profiles/k3s\"\n\t\t\t\t]\n\t\t\t}\n\t\t]\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="WriteJSON\n\t{\n\t\t\"type\": \"sync\",\n\t\t\"status\": \"Success\",\n\t\t\"status_code\": 200,\n\t\t\"operation\": \"\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": [\n\t\t\t{\n\t\t\t\t\"name\": \"incus-ui.crt\",\n\t\t\t\t\"type\": \"client\",\n\t\t\t\t\"restricted\": false,\n\t\t\t\t\"projects\": [],\n\t\t\t\t\"certificate\": \"-----BEGIN CERTIFICATE-----\\nMIIDOjCCAiKgAwIBAgIQARIBdACCADkAOAASAAMkBjANBgkqhkiG9w0BAQUFADBZ\\nMQswCQYDVQQGEwJBVTETMBEGA1UECBMKU29tZS1TdGF0ZTE1MDMGA1UEChMsSW5j\\ndXMgVUkgMTkyLjE2OC4yMDAuMjQwIChCcm93c2VyIEdlbmVyYXRlZCkwHhcNMjQw\\nNjI5MTEyNzEyWhcNMjcwMzI2MTEyNzEyWjBZMQswCQYDVQQGEwJBVTETMBEGA1UE\\nCBMKU29tZS1TdGF0ZTE1MDMGA1UEChMsSW5jdXMgVUkgMTkyLjE2OC4yMDAuMjQw\\nIChCcm93c2VyIEdlbmVyYXRlZCkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK\\nAoIBAQCOEZXU1DDErw/ftWLL4auiEBaXU156KiabPetjeKk7gi59q4/iBvW/QZWw\\nXvdTHj1ju/Svbl+V4u0a+AmH4bEpUcN2+Ss6WCuW8kI743wcymSSHQ6TyK41ML6H\\nFIjvFDgUnVNOegprYQCSz5cqhq7rZJewsI1V2IGky9Hf9jHNtL4CJ7ak2oi6evL9\\ne2t9zO3OmB1HW16ExIKFRXdj0/2hLXFrel/bQCqoG4uYtWqqftoAo/3k9WdDzSMn\\nF6Ch1ZDgKJ/SukT7In2eRfwvReC5jD0oYE/gJtDVAgyStpPTXa6Wtk9mP8opQMzG\\nMBILUn1MHHOXl0teD5SDSupqd9ufAgMBAAEwDQYJKoZIhvcNAQEFBQADggEBAAb4\\nMLoql96sUAZvNr+3eIsILk5k18N8ZAklC5HYVhdAUOPXtxxxviMdYric+g1fxA2E\\nePQ92ZgDy1vRzQNma06McCgw656XmeSwhr8BJnGOPxRIDdJUtmpH7NCuMmfLvvxe\\npdHUFbBKKpHGyCy/NTuBSB0EM6/Ona335T4hKoAURAuKQWYU0sz7oWBmFeKyvxay\\nXcfFjeMLrdy03VruOvCdVxevSHTwYK57ZD3SGF49vHxJluYw1oTvKd+nnluLkLny\\nw74Qo7NI7TBnvb2J3vQ7WYeAouwSXvj+Et7qpzrSdRsZHRC4cAsZRusKVffTkOgo\\nDE/I6sGw8weCK6VSPgc=\\n-----END CERTIFICATE-----\\n\",\n\t\t\t\t\"description\": \"\",\n\t\t\t\t\"fingerprint\": \"a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63\"\n\t\t\t}\n\t\t]\n\t}" http_code=200
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58124" method=GET protocol=tls url="/1.0/events?type=operation,lifecycle&all-projects=true" username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Event listener server handler started" id=a2631acd-1571-48ce-b2d6-b4e738010b46 local="192.168.200.240:8443" remote="192.168.200.35:58124"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url=/1.0 username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Handling API request" ip="192.168.200.35:58110" method=GET protocol=tls url="/1.0/profiles?project=default&recursion=1" username=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: time="2024-07-20T13:51:19+02:00" level=debug msg="Matched trusted cert" fingerprint=a2450b15072ea10651b6a63a14cc700aa24b0577c2630643ae38370bf9815f63 subject="O=Incus UI 192.168.200.240 (Browser Generated),ST=Some-State,C=AU"
Jul 20 13:51:19 stargazer incusd[1083]: incusd: src/vfs.c:802: vfsDatabaseRead: Assertion `amount == (int)page_size' failed.

[stgraber]

Can you show ls -lh /var/lib/incus/database/global/ on a broken system?

We haven't heard of a database corruption case in over a year (including LXD in that too), so having it be somewhat reproducible is very very odd. In any case, recovery should normally be as easy as removing the most recent segment file, there should never be a situation where the entire database is unreadable as dqlite works by building up segments, making it trivial to revert, then doing new baseline snapshots every so often and keeping around multiple snapshots.

So the very few times where corruption happens, removing the most recent snapshot should get you back online and only cause a few minutes of database data loss (if any). If somehow you got all your segments to be corrupted, deleting them will have dqlite go from the latest snapshot. If that snapshot is corrupted, you can delete it and have it start from the previous snapshot.

Having the database be completely dead would require all segment files and all snapshots to be corrupted, at which point you likely have bigger problems with your system :)

@freeekanayaka

[project0]

It just happened again (sorry for the late response, but i am using the machine occassionally). I am not sure what is happening in my case :-(,

I have tried to delete the most recent segment files, it did work for a short amount of time until i tried to create a new VM and it crashed while selecting the image (via incus-UI). Second try with deleting more segment files did seem to work, but its by far no solution as i have now a inconsistent state (vm is not existing anymore, but is still configured and cannot be deleted).

However, here is a state of the broken database files: db.zip

[project0]

I think i found the problem. Apparently incus is dynamically linked in archlinux to cowsql and the packages is not updated to recent version :-( (therefore it still hints to src/vfs.c:802).

[project0]

i guess we can close it, still bad the bug fix depends on OS package distribution. :-(