Cluster member preference in profiles and projects

[yokem55]

Required information

• Distribution: Ubuntu
• Distribution version: 18.04.2
• The output of "lxc info":

  config:
  cluster.https_address: [REDACTED]
  cluster.offline_threshold: "120"
  core.https_address: [REDACTED]
  core.trust_password: true
  api_extensions:
  - storage_zfs_remove_snapshots
  - container_host_shutdown_timeout
  - container_stop_priority
  - container_syscall_filtering
  - auth_pki
  - container_last_used_at
  - etag
  - patch
  - usb_devices
  - https_allowed_credentials
  - image_compression_algorithm
  - directory_manipulation
  - container_cpu_time
  - storage_zfs_use_refquota
  - storage_lvm_mount_options
  - network
  - profile_usedby
  - container_push
  - container_exec_recording
  - certificate_update
  - container_exec_signal_handling
  - gpu_devices
  - container_image_properties
  - migration_progress
  - id_map
  - network_firewall_filtering
  - network_routes
  - storage
  - file_delete
  - file_append
  - network_dhcp_expiry
  - storage_lvm_vg_rename
  - storage_lvm_thinpool_rename
  - network_vlan
  - image_create_aliases
  - container_stateless_copy
  - container_only_migration
  - storage_zfs_clone_copy
  - unix_device_rename
  - storage_lvm_use_thinpool
  - storage_rsync_bwlimit
  - network_vxlan_interface
  - storage_btrfs_mount_options
  - entity_description
  - image_force_refresh
  - storage_lvm_lv_resizing
  - id_map_base
  - file_symlinks
  - container_push_target
  - network_vlan_physical
  - storage_images_delete
  - container_edit_metadata
  - container_snapshot_stateful_migration
  - storage_driver_ceph
  - storage_ceph_user_name
  - resource_limits
  - storage_volatile_initial_source
  - storage_ceph_force_osd_reuse
  - storage_block_filesystem_btrfs
  - resources
  - kernel_limits
  - storage_api_volume_rename
  - macaroon_authentication
  - network_sriov
  - console
  - restrict_devlxd
  - migration_pre_copy
  - infiniband
  - maas_network
  - devlxd_events
  - proxy
  - network_dhcp_gateway
  - file_get_symlink
  - network_leases
  - unix_device_hotplug
  - storage_api_local_volume_handling
  - operation_description
  - clustering
  - event_lifecycle
  - storage_api_remote_volume_handling
  - nvidia_runtime
  - container_mount_propagation
  - container_backup
  - devlxd_images
  - container_local_cross_pool_handling
  - proxy_unix
  - proxy_udp
  - clustering_join
  - proxy_tcp_udp_multi_port_handling
  - network_state
  - proxy_unix_dac_properties
  - container_protection_delete
  - unix_priv_drop
  - pprof_http
  - proxy_haproxy_protocol
  - network_hwaddr
  - proxy_nat
  - network_nat_order
  - container_full
  - candid_authentication
  - backup_compression
  - candid_config
  - nvidia_runtime_config
  - storage_api_volume_snapshots
  - storage_unmapped
  - projects
  - candid_config_key
  - network_vxlan_ttl
  - container_incremental_copy
  - usb_optional_vendorid
  - snapshot_scheduling
  - container_copy_project
  - clustering_server_address
  - clustering_image_replication
  - container_protection_shift
  - snapshot_expiry
  - container_backup_override_pool
  - snapshot_expiry_creation
  - network_leases_location
  - resources_cpu_socket
  - resources_gpu
  - resources_numa
  - kernel_features
  - id_map_current
  - event_location
  - storage_api_remote_volume_snapshots
  - network_nat_address
  - container_nic_routes
  - rbac
  - cluster_internal_copy
  - seccomp_notify
  - lxc_features
  - container_nic_ipvlan
  api_status: stable
  api_version: "1.0"
  auth: trusted
  public: false
  auth_methods:
  - tls
  environment:
  addresses:
  [REDACTED]
  architectures:
  - x86_64
  - i686
  certificate: |
  -----BEGIN CERTIFICATE-----
  [REDACTED]
  -----END CERTIFICATE-----
  certificate_fingerprint: [REDACTED]
  driver: lxc
  driver_version: 3.1.0
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
  netnsid_getifaddrs: "false"
  seccomp_listener: "false"
  shiftfs: "false"
  uevent_injection: "false"
  unpriv_fscaps: "true"
  kernel_version: 4.15.0-50-generic
  lxc_features:
  mount_injection_file: "true"
  network_gateway_device_route: "false"
  network_ipvlan: "false"
  network_l2proxy: "false"
  seccomp_notify: "false"
  project: default
  server: lxd
  server_clustered: true
  server_name: [REDACTED]
  server_pid: 4902
  server_version: "3.13"
  storage: btrfs
  storage_version: "4.4"

  It would be nice to have the ability to have node preferences for certain profiles and/or projects in a lxd cluster. That way we could have a 3-4 node cluster with different performance characteristics and associate certain types of containers to certain nodes while still maintaining a clustering environment. For now, we are just manually moving things around but that gets tedious after a bit.

[stgraber]

Marking it as a maybe for now. This may be a reasonable fit for a project config key but could also get way more complex than we'd like. For now, we expect most of those cases can be manually handled with --target during creation.

[tomponline]

The instance placement scriptlet feature could help achieve what is desired here.

[stgraber]

Indeed. I think that between what we added to restrict projects to specific cluster member groups and the placement scriplet that can make the decision based on the request and additional config, this should be handled now.