if a subnet has multiple static routes, the network interfaces file is invalid

ubuntu-server-builder commented 1 year ago

This bug was originally filed in Launchpad as LP: #1667863

Launchpad details

affected_projects = ['curtin', 'maas']
assignee = None
assignee_name = None
date_closed = 2018-05-26T04:17:25.480217+00:00
date_created = 2017-02-25T00:42:13.744464+00:00
date_fix_committed = None
date_fix_released = None
id = 1667863
importance = undecided
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1667863
milestone = None
owner = mpontillo
owner_name = Mike Pontillo
private = False
status = expired
submitter = jeremy-huylebroeck
submitter_name = Jeremy Huylebroeck
tags = []
duplicates = []

Launchpad user Jeremy Huylebroeck(jeremy-huylebroeck) wrote on 2017-02-25T00:42:13.744464+00:00

I have multiple subnets, each has an additional custom static route.

those subnets are used by different bridges on the same node.

example: brAdm (on interface enp9s0) - subnet 172.30.72.128/25 - static route 172.30.72.0/21 gw 172.30.72.129 brPublic (on interface ens9.2002) - subnet 172.30.80.128/25 - static route 172.30.80.0/21 gw 172.30.80.129

the resulting pre-up and post-up lines in /etc/network/interfaces are malformed, which creates then the wrong routing table.

It seems the pre-down of one route and the post-up of the next route are not separated by a newline.

See below:

post-up route add -net 172.30.80.0 netmask 255.255.248.0 gw 172.30.80.129 metric 0 || true pre-down route del -net 172.30.80.0 netmask 255.255.248.0 gw 172.30.80.129 metric 0 || truepost-up route add -net 172.30.72.0 netmask 255.255.248.0 gw 172.30.72.129 metric 0 || true pre-down route del -net 172.30.72.0 netmask 255.255.248.0 gw 172.30.72.129 metric 0 || true

Here's the entire resulting network configuration for reference. note that a bunch of other bridge interfaces are created, but not used on this machine, so not configured.

cat /etc/network/interfaces auto lo iface lo inet loopback dns-nameservers 172.30.72.130 dns-search r16maas.os maas

auto enp9s0 iface enp9s0 inet manual mtu 9000

auto ens9 iface ens9 inet manual mtu 9000

auto brAdm iface brAdm inet static address 172.30.72.132/25 hwaddress ether 08:9e:01:ab:fc:f6 bridge_ports enp9s0 bridge_fd 15 mtu 9000

auto brData iface brData inet manual hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brExt iface brExt inet manual hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brInt iface brInt inet manual hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brPublic iface brPublic inet static address 172.30.80.132/25 gateway 172.30.80.129 hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brStoClu iface brStoClu inet manual hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brStoData iface brStoData inet manual hwaddress ether 00:02:c9:ce:7c:16 bridge_ports ens9.0 bridge_fd 15 mtu 9000

auto brAdm.52 iface brAdm.52 inet manual vlan_id 52 mtu 1500 vlan-raw-device brAdm

auto ens9.0 iface ens9.0 inet manual mtu 9000 vlan-raw-device ens9 post-up route add -net 172.30.80.0 netmask 255.255.248.0 gw 172.30.80.129 metric 0 || true pre-down route del -net 172.30.80.0 netmask 255.255.248.0 gw 172.30.80.129 metric 0 || truepost-up route add -net 172.30.72.0 netmask 255.255.248.0 gw 172.30.72.129 metric 0 || true pre-down route del -net 172.30.72.0 netmask 255.255.248.0 gw 172.30.72.129 metric 0 || true source /etc/network/interfaces.d/*.cfg

route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.30.72.128 255.255.255.128 U 0 0 0 brAdm 172.30.80.128 255.255.255.128 U 0 0 0 brPublic

ifconfig brAdm Link encap:Ethernet HWaddr 08:9e:01:ab:fc:f6 inet addr:172.30.72.132 Bcast:172.30.72.255 Mask:255.255.255.128 inet6 addr: fe80::a9e:1ff:feab:fcf6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:15029 errors:0 dropped:0 overruns:0 frame:0 TX packets:1447 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7393978 (7.3 MB) TX bytes:182411 (182.4 KB)

brAdm.52 Link encap:Ethernet HWaddr 08:9e:01:ab:fc:f6 inet6 addr: fe80::a9e:1ff:feab:fcf6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7885 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:398943 (398.9 KB) TX bytes:488 (488.0 B)

brData Link encap:Ethernet HWaddr 00:02:c9:ce:7c:16 inet6 addr: fe80::202:c9ff:fece:7c16/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:152 (152.0 B) TX bytes:648 (648.0 B)

brExt Link encap:Ethernet HWaddr ca:48:e5:20:50:fa inet6 addr: fe80::c848:e5ff:fe20:50fa/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)

brInt Link encap:Ethernet HWaddr fa:63:19:24:ed:80 inet6 addr: fe80::f863:19ff:fe24:ed80/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)

brPublic Link encap:Ethernet HWaddr 00:02:c9:ce:7c:16 inet addr:172.30.80.132 Bcast:172.30.80.255 Mask:255.255.255.128 inet6 addr: fe80::9840:fbff:fe1f:b2b4/64 Scope:Link UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:258 (258.0 B)

brStoClu Link encap:Ethernet HWaddr c2:09:5a:67:d3:da inet6 addr: fe80::c009:5aff:fe67:d3da/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)

brStoData Link encap:Ethernet HWaddr 0e:24:f0:9b:e5:ea inet6 addr: fe80::c24:f0ff:fe9b:e5ea/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)

enp9s0 Link encap:Ethernet HWaddr 08:9e:01:ab:fc:f6 inet6 addr: fe80::a9e:1ff:feab:fcf6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:18629 errors:0 dropped:67 overruns:0 frame:0 TX packets:1454 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7960838 (7.9 MB) TX bytes:190759 (190.7 KB) Interrupt:18 Memory:dfd00000-dfd20000

ens9 Link encap:Ethernet HWaddr 00:02:c9:ce:7c:16 inet6 addr: fe80::202:c9ff:fece:7c16/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:9747 errors:0 dropped:67 overruns:0 frame:0 TX packets:154 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:658955 (658.9 KB) TX bytes:15148 (15.1 KB)

ens9.0 Link encap:Ethernet HWaddr 00:02:c9:ce:7c:16 UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:152 (152.0 B) TX bytes:1156 (1.1 KB)

lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:163 errors:0 dropped:0 overruns:0 frame:0 TX packets:163 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:12104 (12.1 KB) TX bytes:12104 (12.1 KB)

ubuntu-server-builder commented 1 year ago

Launchpad user Jeremy Huylebroeck(jeremy-huylebroeck) wrote on 2017-02-28T23:45:02.043065+00:00

want to add 2 points:

I go around this bug by modifying the /etc/maas/preseeds/curtin_userdata_XXXX for my nodes with the following command in the late_commands section.

my_fix_static_route: curtin in-target -- sh -c "sed -i.bak s/'truepost-up route add'/'true\npost-up route add'/g /etc/network/interfaces"
I think that having all the post-up / pre-down commands at the end of the interfaces files is not appropriate. It should be instead IMHO up/down commands in each respective interface with the subnet with that static route.

ubuntu-server-builder commented 1 year ago

Launchpad user Andres Rodriguez(andreserl) wrote on 2017-06-27T17:15:54.761229+00:00

Hi Jeremy,

Thanks for your bug report. I was wondering if this bug is still valid for you and wondering if you could provide a bit more information. Can you get the curtin configuration:

maas node get-curtin-config <node's system id>

Also, can you confirm what version of MAAS & what version of curtin are you using (dpkg -l | grep curtin, etc).

Thanks.

ubuntu-server-builder commented 1 year ago

Launchpad user Jeremy Huylebroeck(jeremy-huylebroeck) wrote on 2017-06-27T18:27:39.669567+00:00

I have to check if the bug still exists. My fix made me forget about it... :)

I have updated Maas since I reported the bug. In February I think I was basically using the bleeding edge version because I was testing some integration with OPNFV/Juju.

here's what I have now

$ maas ubuntu version read Success. Machine-readable output follows: {"version": "2.1.5+bzr5596-0ubuntu1", "capabilities": ["networks-management", "static-ipaddresses", "ipv6-deployment-ubuntu", "devices-management", "storage-deployment-ubuntu", "network-deployment-ubuntu", "bridging-interface-ubuntu", "bridging-automatic-ubuntu"], "subversion": "16.04.1"}

The get-curtin-config command doesn't exist though in my setup. Am I missing something?

Here's the current curtin version installed: $ dpkg -l | grep curtin ii curtin-common 0.1.0~bzr470-0ubuntu1~16.04.1 all Library and tools for curtin installer ii python3-curtin 0.1.0~bzr470-0ubuntu1~16.04.1 all Library and tools for curtin installer

and the maas packages: $ dpkg -l | grep maas ii maas 2.1.5+bzr5596-0ubuntu1~16.04.1 all "Metal as a Service" is a physical cloud and IPAM ii maas-cli 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS client and command-line interface ii maas-common 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS server common files ii maas-dhcp 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS DHCP server ii maas-dns 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS DNS server ii maas-proxy 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS Caching Proxy ii maas-rack-controller 2.1.5+bzr5596-0ubuntu1~16.04.1 all Rack Controller for MAAS ii maas-region-api 2.1.5+bzr5596-0ubuntu1~16.04.1 all Region controller API service for MAAS ii maas-region-controller 2.1.5+bzr5596-0ubuntu1~16.04.1 all Region Controller for MAAS ii python3-django-maas 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS server Django web framework (Python 3) ii python3-maas-client 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS python API client (Python 3) ii python3-maas-provisioningserver 2.1.5+bzr5596-0ubuntu1~16.04.1 all MAAS server provisioning libraries (Python 3)

ubuntu-server-builder commented 1 year ago

Launchpad user Mike Pontillo(mpontillo) wrote on 2018-03-23T22:43:17.953946+00:00

Adding cloud-init. This looks like an issue with how the netplan gets rendered.

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2018-03-26T13:14:59.651050+00:00

Hi, Curtin is no longer involved in rendering of Ubuntu network config, so I've marked its task as Invalid.

Please attach the network configuration that MAAS sent. Without this we are not able to do much in cloud-init.

In order to further investigate this bug we'll need to get some information about the node. Please collect and attach the storage configuration for the node. You can get this information with:

maas 2.0 via cli maas machine get-curtin-config
Web UI: On the node details page in the installation output section at the bottom of the page

When you've provided this information, please set back to 'New' for cloud-init.

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2018-05-26T04:17:25.287790+00:00

[Expired for cloud-init because there has been no activity for 60 days.]

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2018-05-26T04:17:27.268262+00:00

[Expired for MAAS because there has been no activity for 60 days.]

canonical / cloud-init

if a subnet has multiple static routes, the network interfaces file is invalid #2814