kkaempf
commented
2 months ago Can I express fixed IP assignments somehow ? 🤔
Closed anmazzotti closed 1 month ago
kkaempf
commented
2 months ago Can I express fixed IP assignments somehow ? 🤔
anmazzotti
commented
2 months ago The MachineInventory now carries complete information about the IPs and network config, so that on reconcile it's easier to manage everything.
apiVersion: v1
items:
- apiVersion: elemental.cattle.io/v1beta1
kind: MachineInventory
metadata:
annotations:
elemental.cattle.io/network.applied: "false"
name: test-btrfs-d47ca4cf-e95e-4e43-939b-e5bce460fbfc
namespace: fleet-default
spec:
ipAddressClaims:
inventory-ip:
apiVersion: ipam.cluster.x-k8s.io/v1beta1
kind: IPAddressClaim
name: test-btrfs-d47ca4cf-e95e-4e43-939b-e5bce460fbfc-inventory-ip
namespace: fleet-default
uid: ac9d9629-c545-41ec-93f0-e7a59b2fcfd2
ipAddressPools:
- ipPoolRef:
apiGroup: ipam.cluster.x-k8s.io
kind: InClusterIPPool
name: elemental-inventory-pool
name: inventory-ip
network:
config:
dns-resolver:
config:
search: []
server:
- 192.168.122.1
interfaces:
- description: Main-NIC
ipv4:
address:
- ip: '{inventory-ip}'
prefix-length: 24
dhcp: false
enabled: true
ipv6:
enabled: false
name: enp1s0
state: up
type: ethernet
routes:
config:
- destination: 0.0.0.0/0
metric: 150
next-hop-address: 192.168.122.1
next-hop-interface: enp1s0
table-id: 254
ipAddresses:
inventory-ip: 192.168.122.150
tpmHash: 759d02b8cdfad743082ac528214a5b00cb3d6079aaa2b6bb35d87209e36f1fdb
status:
conditions:
- lastTransitionTime: "2024-07-04T08:24:48Z"
message: NetworkConfig is ready
reason: ReconcilingNetworkConfig
status: "True"
type: NetworkConfigReadyCan I express fixed IP assignments somehow ? 🤔
Not really. What are you trying to accomplish here? There should be no particular attachment to one IP rather than another one, this is why we are supporting an IPAM provider. You could theoretically define IPPools with 1 IP only, but that will also force you to one registration per machine and completely defeat what we are implementing here.
kkaempf
commented
2 months ago Can I express fixed IP assignments somehow ? 🤔
Not really. What are you trying to accomplish here?
I wonder how IP assignments to cluster nodes survive across reboots.
anmazzotti
commented
2 months ago I wonder how IP assignments to cluster nodes survive across reboots.
The network config is persisted through a yip file in /oem
For example this is included post-installation. Note this will also run in recovery mode:
name: Apply network config
stages:
initramfs:
- files:
- path: /oem/network/applied-config.yaml
permissions: 384
owner: 0
group: 0
content: |
dns-resolver:
config:
search: []
server:
- 192.168.122.1
interfaces:
- description: Main-NIC
ipv4:
address:
- ip: '192.168.122.150'
prefix-length: 24
dhcp: false
enabled: true
ipv6:
enabled: false
name: enp1s0
state: up
type: ethernet
routes:
config:
- destination: 0.0.0.0/0
metric: 150
next-hop-address: 192.168.122.1
next-hop-interface: enp1s0
table-id: 254
encoding: ""
ownerstring: ""
directories:
- path: /oem/network
permissions: 448
owner: 0
group: 0
if: '[ ! -f /oem/network/applied-config.yaml ]'
network.before:
- commands:
- nmstatectl apply /oem/network/applied-config.yamlA lot happened in this PR.
The current status of things is that it does use nmstatectl to generate *.nmconnection files during the Elemental installation phase.
For example, given a MachineRegistration such as:
apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
name: fire-nodes
namespace: fleet-default
spec:
machineName: test-btrfs-${System Information/UUID}
config:
network:
ipAddresses:
inventory-ip:
apiGroup: ipam.cluster.x-k8s.io
kind: InClusterIPPool
name: elemental-inventory-pool
config:
dns-resolver:
config:
server:
- 192.168.122.1
search: []
routes:
config:
- destination: 0.0.0.0/0
next-hop-interface: enp1s0
next-hop-address: 192.168.122.1
metric: 150
table-id: 254
interfaces:
- name: enp1s0
type: ethernet
description: Main-NIC
state: up
ipv4:
enabled: true
dhcp: false
address:
- ip: "{inventory-ip}"
prefix-length: 24
ipv6:
enabled: falseWill lead to the following yip file (/oem/94_custom.yaml) to apply network configuration:
name: Apply network config
stages:
initramfs:
- files:
- path: /etc/NetworkManager/system-connections/Wired connection 1.nmconnection
permissions: 384
owner: 0
group: 0
content: |
[connection]
id=Wired connection 1
uuid=d26b4ae4-d525-3cbf-a557-33feb60343c0
type=ethernet
autoconnect-priority=-999
interface-name=enp1s0
timestamp=1722339542
[ethernet]
[ipv4]
address1=192.168.122.150/24
dhcp-timeout=2147483647
dns=192.168.122.1;
dns-options=
dns-priority=40
method=manual
route1=0.0.0.0/0,192.168.122.1,150
route1_options=table=254
[ipv6]
addr-gen-mode=eui64
dhcp-timeout=2147483647
method=disabled
[proxy]
[user]
nmstate.interface.description=Main-NIC
encoding: ""
ownerstring: ""
if: '[ -f /run/elemental/active_mode ]'
Part of #756
This PR should introduce network config support. Note that the machine registration and reset flows still require DHCP support, however this PR removes the need of "infinite leases".
The elemental-operator will be able to use any CAPI IPAM provider to reserve and assign static IPAddresses to each MachineInventory.
The elemental-register will be able to ask for a network config to be applied. While the actual config in the NetworkTemplate is schemaless, this PR implements nmstate only.
This will require the OS to also have nmstate installed, so that the config can be applied.
Note that it is possible to define a NetworkTemplate on both MachineRegistration and MachineSelector. The latter will be applied during the MachineInventory adoption phase, so that it is possible to define IPPools per Cluster (rather per Registration only).
A registration example: