Webmin on Ubuntu Server 18.04LTS: Network settings not working with Netplan

[extremeskillz]

Ubuntu uses Netplan on server side now since 17.10 I believe and I am having issues with Webmin not able to edit my network configs and keeping them. The active on boot window is where I put my static config in and when I apply the network changes it does not apply to the Active now side. No change after a reboot as well. This is definitely a bug. This worked fine on Ubuntu 14.04lts. This limits my upgrade process to the latests OS since I use Webmin on all Ubuntu servers for a GUI.

[jcameron]

Known issue, which is being actively worked on.

[extremeskillz]

Thanks for the update. For now I will keep the latest deployments on Ubuntu server 16.04lts since webmin network settings work on the system with no issues.

[Sopor]

# ifupdown has been replaced by netplan(5) on this system.  See
# /etc/netplan for current configuration.
# To re-enable ifupdown on this system, you can run:
#    sudo apt install ifupdown

[extremeskillz]

I saw that but I wanted to state the bug with webmin using netplan since that is the way Ubuntu is going so Webmin can work with Netplan and other OS's using it. Webmin currently work as it should on Ubuntu 16.04lts. I hope the support for 18.04LTS comes in the near future.

[gametimesav]

Still not love ... I didn't even know about this till now. Was doing a fresh install, and went to setup the firewall. Then started looking at my main server and seen it was a problem also. Beem kinda looking around for a few days for whats up. Didn't know at first if it was something i did or what ... Happy to see it is a problem ... lol ... Please fix this ... Webmin has always worked great on ubuntu ...

[iliajie]

@gametimesav What is the Webmin version that you have installed, where it's still not working?

[gametimesav]

 The networking part ... Non of the firewall settings work

Operating system Ubuntu Linux 18.04 Webmin version 1.890 Theme version Authentic Theme https://github.com/authentic-theme/authentic-theme 19.19

On Fri, Jul 20, 2018 at 2:09 AM Ilia Rostovtsev notifications@github.com wrote:

@gametimesav https://github.com/gametimesav What is the Webmin version that you have installed, where it's still not working?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webmin/webmin/issues/930#issuecomment-406498715, or mute the thread https://github.com/notifications/unsubscribe-auth/AZJs4mXecrRbrF6vGVRq5j9mBDJMc8Mjks5uIXP9gaJpZM4U7sbT .

[gametimesav]

Rules file /etc/ip6tables.up.rules HTTP/1.0 500 Perl execution failed Server: MiniServ/1.890 Date: Fri, 20 Jul 2018 13:41:31 GMT Content-type: text/html; Charset=iso-8859-1 Connection: closeError - Perl execution failed

Undefined subroutine &net::get_interface_defs called at debian-linux-lib.pl line 63.

On Fri, Jul 20, 2018 at 9:40 AM GameTime SportsBar gametimesav@gmail.com wrote:

The networking part ... Non of the firewall settings work

Operating system Ubuntu Linux 18.04 Webmin version 1.890 Theme version Authentic Theme https://github.com/authentic-theme/authentic-theme 19.19

On Fri, Jul 20, 2018 at 2:09 AM Ilia Rostovtsev notifications@github.com wrote:

@gametimesav https://github.com/gametimesav What is the Webmin version that you have installed, where it's still not working?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webmin/webmin/issues/930#issuecomment-406498715, or mute the thread https://github.com/notifications/unsubscribe-auth/AZJs4mXecrRbrF6vGVRq5j9mBDJMc8Mjks5uIXP9gaJpZM4U7sbT .

[p-schneider]

In (development) Version 1.890 webmin is not able to properly parse files like

network:
    renderer: networkd
    ethernets:
        ens160:
            addresses: [123.123.123.123/29, '1234:1234:1234:1234::1:2/112']
            gateway4: 123.123.123.111
            gateway6: 1234:1234:1234:1234::1:1
            dhcp4: false
            dhcp6: false
            optional: true
            nameservers:
                addresses: [123.123.123.111, '1234:1234:1234:1234::1:1']
                search: [domain.tld, secondary.search.doma.in]
    version: 2

in /etc/netplan/*.yaml but without double or single quotes it wouldn't be valid yaml so netplan wouln't be able to parse what webmin generates. (Using static IPv6 setup on Ubuntu 18.04, it doesn't seem to matter if I use "double quotes" or 'single quotes')

[jcameron]

The firewall issue should be fixed with this commit : https://github.com/webmin/webmin/commit/e0621384a9da116b1898ea4365a659bc82df9f6b

[jcameron]

Also, just checked in a couple of fixes for the quoted addresses.

[p-schneider]

Is there already a .deb we can use to test the fixes provided in the latest commits? (I saw that the last modified changed for the files in https://download.webmin.com/devel/deb/ but the version stayed at 1.890 and the 1.890 tag is 6 commits behind master so i assume the last modified just came from some cron job last night that just reuploaded the same files.)

[jcameron]

There's no .deb with the fixes yet, but they will be in 1.891.

[extremeskillz]

Looks like there is progress on this. I will be firing up a fresh 18.04LTS server VM to test 1.891 when it is released. For now we kept our latest servers on Ubuntu 16.04lts with webmin 1.890 and they are fine and configure as they should with Webmin.

[gametimesav]

Networking firewall was working in beta-1 theme .. IDK ... then in beta-2 it stopped again ... however, read user mail is working now ... Just did a full update today .... FYI

On Tue, Aug 7, 2018 at 3:12 PM extremeskillz notifications@github.com wrote:

Looks like there is progress on this. I will be firing up a fresh 18.04LTS server VM to test 1.891 when it is released. For now we kept our latest servers on Ubuntu 16.04lts with webmin 1.890 and they are fine and configure as they should with Webmin.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webmin/webmin/issues/930#issuecomment-411168190, or mute the thread https://github.com/notifications/unsubscribe-auth/AZJs4vUh9cmecKgqGZ7BuDP8KLZnOgvgks5uOeZ_gaJpZM4U7sbT .

[jcameron]

Have you tried Webmin 1.891 from http://www.webmin.com/devel.html ? It fixes some of these Ubuntu 18 issues.

[gametimesav]

No .. always used apt to install it ..... sarge contrib ... how or can i cant it here to the dev version? and how?

On Thu, Aug 9, 2018 at 12:53 AM Jamie Cameron notifications@github.com wrote:

Have you tried Webmin 1.891 from http://www.webmin.com/devel.html ? It fixes some of these Ubuntu 18 issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webmin/webmin/issues/930#issuecomment-411636149, or mute the thread https://github.com/notifications/unsubscribe-auth/AZJs4orr0Pawt1PIrKwDUR7khTftY5bnks5uO8BAgaJpZM4U7sbT .

[p-schneider]

wget https://download.webmin.com/devel/deb/webmin_current.deb
sudo dpkg -i webmin_current.deb

For me it still doesn't fully work in 1.891, it still looks like this: (see my comment from 20 days ago)

[jcameron]

Ok, turns out there is a separate bug that is causing this :-(

It is fixed by this patch though : https://github.com/webmin/webmin/commit/c24070ff0cd8e1713bb43f102117a888d1633da6

[extremeskillz]

Just found out the recent release is causing issues on Ubuntu 16.04LTS network settings as well. What is happening is what is going on with 18.04lts. So same symptom. You update the network settings from DHCP to static, save and apply, and nothing happens. 16.04lts does not use Netplan. Only 17.10 and forward. So right now there is the same issues with two OS releases. I wanted to let you know.

[p-schneider]

Ok, turns out there is a separate bug that is causing this :-(

It is fixed by this patch though : c24070f

Even with that patch (patched manually on top of a 1.891 install) the ipv6 address is still interpreted as IPv4 instead of being listed in the IPv6-address column.

[extremeskillz]

Thanks for the update. Right now I configured my latest 16.04LTS releaseses manually and that is holding and being interpeted by webmin if you view it but don't make changes. Skipping 18.04LTS for now.

[swelljoe]

Current issues I'm seeing, as of 1.893 (not sure if there are changes since 1.893 was rolled, I just noticed it):

• Nameservers are being turned into interface addresses when saving (during Virtualmin installation).
• Interfaces are being duplicated, maybe multiple times per save.

Here's two config files, the first is roughly what Webmin is starting with, and the second is what it looks like after:

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        eth0:
            addresses:
            - 162.243.170.243/24
            - 2604:A880:0400:00D0:0000:0000:02ED:9001/64
            - 10.10.0.6/16
            gateway4: 162.243.170.1
            gateway6: 2604:A880:0400:00D0:0000:0000:0000:0001
            match:
                macaddress: d6:bc:fe:6d:20:b6
            nameservers:
                addresses:
                - 67.207.67.3
                - 67.207.67.2
                search: []
            set-name: eth0

And, after:

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        eth0:
            addresses: ['206.189.184.79/20','2604:A880:0400:00D1:0000:0000:06E9:E001/64','2604:A880:0400:00D1:0000:0000:06E9:E001/64','10.10.0.5/16','67.207.67.2/32','67.207.67.3/32','206.189.184.79/20','10.10.0.5/16','67.207.67.2/32','67.207.67.3/32','206.189.184.79/20','10.10.0.5/16','67.207.67.2/32','67.207.67.3/32']
            gateway4: 206.189.176.1
            gateway6: 2604:a880:0400:00d1:0000:0000:0000:0001
            nameservers:
                addresses: [127.0.0.1,127.0.0.53]
            match: 
                macaddress: de:81:5f:eb:0e:bc
                set-name: eth0

Note the nameserver addresses are now listed (multiple times) in the interfaces addresses section, and the actual addresses have also been duplicated.

I killed the Digital Ocean droplet I had granted you access to previously, but I can get you access to the new one, if you need to see where this happens.

Edit: Note these are from two different instances, so the network IPs are different, as I didn't make a backup of the netplan config before running the Virtualmin installer, so just spun up a new instance with the same config.

[jcameron]

@swelljoe - you might want to try the latest checked-in code and see if that helps. I will build a 1.894 devel version this week once I get my regular devel environment back online.

[swelljoe]

I looked and didn't see any networking related commits since 1.893 was tagged. But, I'll give it a poke just in case.

[jcameron]

There are some fixes here : https://github.com/webmin/webmin/commits/master/net/netplan-lib.pl

I think they fix the problem with addresses being duplicated, but I will build a new 1.894 devel version to test with on your droplet (if it's still up).

[swelljoe]

Nope, it's still slurping the name servers into interfaces, and repeating interface addresses.

Edit: With 1.894.

[marcelmah]

Would like to have this fixed to (can't program unfortunately), hate defining fixed IP's in text config files.

[bobinson]

much needed fix ...

[swelljoe]

Can y'all poke at Webmin 1.900 and see if the problems you had have been resolved? Jamie thinks it's all been sorted out, but testing might shake out more problems or unsupported syntax.

[swelljoe]

Nope, still borked, but in new ways.

Here's an example of a botched config with 1.900:

network:
  version: 2
  renderer: networkd
  ethernets:
        eth0:
            addresses: ['/32']
            nameservers:
                addresses: [127.0.0.1,127.0.0.53]
                search: [invalid]
            match:
                macaddress: 23:05:44:46:6d:5a
        macaddress: 23:05:44:46:6d:5a
      addresses:
        - 176.249.10.180/24
        - 2a35:c233:0000:1358:0000:0000:0000:0001/64
      gateway4: 176.249.10.1
      gateway6: fe80::1
      nameservers:
        search: [ invalid ]
        addresses:
          - 8.8.8.8
          - 8.8.4.4
          - 2a35:c233::2:53
          - 2a99:c237::1:53

[swelljoe]

The things that I think(?) are wrong:

• Moved interface eth0 addresses value (176.249.10.1, and the IPv6 one) out into a standalone addresses section. Left an empty-ish ['/32'] in the eth0 section.
• Got the search field wrong (not sure what it was supposed to be on this system, but invalid obviously aint it).
• eth0 doesn't actually exist on this system...so, I'm not sure what's going on there. it's actually named enp1s0. This may, or may not, be Webmin's doing. Maybe there's a magical alias feature or something that I don't understand and eth0 can be used. I cleaned it up as best I could, so the customer could get to work, but I'll see if I can reproduce this kind of problem on one of our servers, so we can break it at will.

[jcameron]

I'd be really interested to see what the config looked like before the corruption!

Also, was this triggered by some action in the Webmin UI, or when the internal API to edit an interface was called by some other script?

[swelljoe]

I wish I'd made a backup. I can't seem to reproduce it on other servers that I can tinker with. I've tried on both a Vultr and Digital Ocean VM and while both are now using netplan configs, by default, the installer works successfully and doesn't destroy the configuration, as far as I can tell.

I'll keep an eye out for other folks with problems, but I think we're doing the right thing most of the time. Maybe just a specific kind of YAML trips up the parser. I don't know if it's interesting/useful or not, but when I did the install, the Networking configuration step failed the first time, but then succeeded when I ran it again...so, I think Webmin re-wrote the config in a form that it would be happy with, and when it read it the next time didn't choke on it. (Of course, what it wrote was broken.)

[marcelmah]

I just build a new VM based on Ubuntu 18.04 (mini.iso) to test this.

Installed Webmin (1900) and tested basic configuration (just change DHCP to fixed IP). Config looked fine, reboot also returned the VM in a working state. So basic functionality looks to be ok.

[cengbrecht]

I have rebuilt my Unifi Controller 3 times now because of this. :P I can replicate it everytime.

Need some test data?

[extremeskillz]

Looks like 1900 is released so I will try it and leave feed back accordingly.

[swelljoe]

@cengbrecht yes, if you could paste in the original config (the files in /etc/netplan) and the config that Webmin generates so we can see in what way it is breaking things, that would be extremely helpful.

[cengbrecht]

@swelljoe Here is before. :) This is a temp server:

[cengbrecht]

@swelljoe Sorry, forgot to install webmin. :P There was no change immediately after install. :) However, after adding 9.9.9.9 to the dns list in the network config:

[cengbrecht]

Before netplan Apply: No change after apply. Rebooting.

[cengbrecht]

It would seem that Netplan is not broken on a fresh install.

[cengbrecht]

@swelljoe I am going to add more nic's and ipv6. I only have ipv4 enabled, and no local networking. After adding IPv6, and local networking, the file did not update, however I do have IPv6 in the client. I am not sure what happened to the local networking... ? Also, I am a Git Newb, so please let me know if there's a better practice for the config post, when I tried in the code brackets, it was broken...
Can't seem to see the local networking config, however, I think that is more that I added it post config, and post webmin install. Please let me know if you'd like me to re-test with a fresh install, and with the networking setup before webmin install.

[swelljoe]

It looks like things worked OK...I don't see anything "wrong" with the config Webmin generated. It's differently formatted (and maybe it'd be ideal if it retained the style), but either is valid YAML and looks like legit config.

When posting config files like this, I just use three backticks to open and close the code block. It worked fine for me above for these YAML files.

Anyway, I think we're stuck in a holding pattern until we can find a config that breaks. I know it does break in some circumstances, because I've seen it myself but it was on a client box that I couldn't really break at will, and I didn't have a backup of the original config, and once I cleaned it up Webmin was happy with it. So...

[jcameron]

Yeah, if you can find a config that Webmin re-writes incorrectly, please let me know.

[extremeskillz]

So as of today, 12/5, Webmin 1.9000 will not setup the static IP still. When you go and do it and apply it just keeps it in DHCP and ignores it. I did check my Netplan files and nothing changed. Still an issue. I wanted to leave some feedback. In fact, after a reboot the nic was left down with no config. Had to revert back to DHCP to get it working.

[joeyd]

So as of today, 12/5, Webmin 1.9000 will not setup the static IP still. When you go and do it and apply it just keeps it in DHCP and ignores it. I did check my Netplan files and nothing changed. Still an issue. I wanted to leave some feedback. In fact, after a reboot the nic was left down with no config. Had to revert back to DHCP to get it working.

Exact same issue here with 1.9000.

[joeyd]

For more claification, I have just done a fresh install of Ubuntu 18.04.1 (Official AWS Provided AMI) and Set a FQDN hostname and installed Virtualmin (1.9000) under VPC. Install of Virtualmin was error free.

1. Login to virtualmin as root, create new virtualhost and assign static private IP.
2. Went to Webmin > Networking > Network Configuration > Network Interfaces, Interface did get generated from step 1.
3. Rebooted server, went to Webmin > Networking > Network Configuration > Network and the interface created above is missing and now reporting a new static IP.

Here is the 50-cloud-init.yaml with a fresh install

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        ens5:
            dhcp4: true
            nameservers:
                addresses: [127.0.0.1,127.0.0.53]
                search: [ec2.internal]
            match:
                macaddress: 12:52:c6:ec:32:6a
            set-name: ens5

Here is the 50-cloud-init.yaml after adding virtual server

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        ens5:
            dhcp4: true
            addresses: ['10.0.3.52/24']
            nameservers:
                addresses: [127.0.0.1,127.0.0.53]
                search: [ec2.internal]
            match:
                macaddress: 12:52:c6:ec:32:6a
            set-name: ens5

Here is the 50-cloud-init.yaml after reboot of adding virtual server

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        ens5:
            dhcp4: true
            addresses: ['10.0.3.52/24']
            nameservers:
                addresses: [127.0.0.1,127.0.0.53]
                search: [ec2.internal]
            match:
                macaddress: 12:52:c6:ec:32:6a
            set-name: ens5

If there is any other info I can provide, let me know.

[jcameron]

There is another bug that prevents switching out of DHCP mode properly, fixed by this patch : https://github.com/webmin/webmin/commit/02ad7654087d92f87ca5bbb6998ac81412df5359#diff-1017cdb95b6200e8b2a979089b5b032c

[extremeskillz]

Will that be in the next release?