Run Successfully but does not add Kea DHCP reservations

[WaveZero]

I am running into an interesting problem where the migrate program runs successfully (no error codes or messages) on Windows 11 Pro Build 22631 but the new_config.xml does not have any Kea DHCP reservations from ISC DHCP.

I'm happy to share my opnsense config.xml and the new_config.xml via email for you to validate if that helps.

Here's an example of one of my ISC DHCP reservations:

      <staticmap>
        <mac>a0:3d:6f:66:b8:18</mac>
        <ipaddr>192.168.1.110</ipaddr>
        <hostname>Wave-AIR-CAP3702I-AP01</hostname>
        <descr>Cisco WAP #1</descr>
        <winsserver/>
        <dnsserver/>
        <ntpserver/>
      </staticmap>

OPNsense version: 24.1.4 (amd64)

Thanks for your help.

[EasyG0ing1]

@WaveZero

I'd be happy to look at those config files. Even if you've already completed your migration, they would help me correct the problem for future users.

My email: sims.mike@gmail.com

[WaveZero]

@EasyG0ing1

Thank you, I have emailed you both my config.xml and new_config.xml.

I have yet to migrate my ISC DHCP reservations to Kea yet, happy to test if you manage to work out what the issue is.

Kind regards,

Wave

[EasyG0ing1]

@WaveZero So initial test ... where I just ran it on my mac, it worked just fine. I'm going to check it in Windows and update this post here with what I find.

[EasyG0ing1]

@WaveZero

OK, I posted a new Windows build in the release section but here it is as well. This should work fine.

Windows-Migrate.zip

Make sure you create a new config.xml since it's been a while since you took a backup of your OPNsense environment. You'll be restoring everything that is in config.xml so you'll want to make sure that file is current before running migrate on it.

[WaveZero]

Thanks @EasyG0ing1,

I have sent you an email describing another issue I have encountered.

I can now see the Kea DHCP reservations in the new_config.xml file, however when I try to restore it, it's missing the IP address and MAC address mappings.

[EasyG0ing1]

@WaveZero

When you look at the new_config.xml file and scroll down towards the bottom ... you'll see mappings in there that have this framework:

<reservation uuid="dd505962-7850-4205-88ac-0d8733445b51">
<subnet>662d9330-7ecd-4c53-82b9-01a7066539a3</subnet>
    <ip_Address>192.168.1.1</ip_Address>
    <hw_Address>XX:XX:XX:XX:XX:XX</hw_Address>
    <hostname>hostname</hostname>
    <description>description</description>
</reservation>

Do you see those in your new_config.xml file with IP addresses and everything?

[WaveZero]

@EasyG0ing1

I have sent you the new_config.xml via email for your reference but yes I do see the mappings, however, on restore it's missing both the IP address and MAC address mappings.

[EasyG0ing1]

@WaveZero And you are looking under Services / Kea DHCP [new] / Kea DHCP v4 Then you click on the Reservations tab and you are saying that you have entries in there but that they have no IP or MAC address?

[WaveZero]

@EasyG0ing1 that's correct.

[EasyG0ing1]

@WaveZero I also wanted to mention that your IP Pool seems to have been assigned the entire subnet. While this is not necessarily a problem - because we assume that a static reservation won't be handed out from the pool ... it's generally best practice to define your pool outside of the addresses that you set aside for static reservation. The intent there is to guarantee no overlapping assignments as well as for management clarity etc.

And you have a lot of statics, you might consider dropping your subnet mask to a 23 bit mask which would make your network have these available IP addresses: 192.168.0.1 - 192.168.1.254 with the network address being 192.168.0.0 and the broadcast being 192.168.1.255 With a setup like that, you could define your DHCP pool as being 192.168.1.100 - 192.168.1.254 Then just do your static maps from x.x.0.1 to x.x.1.99 which gives you roughly 350-ish IP addresses while another 150 remain available for dynamic assignments.

Might be too much, but head room is always desireable with DHCP.

[EasyG0ing1]

@WaveZero - Did you create a new config.xml before running the migration and importing the new config?

[EasyG0ing1]

@WaveZero Im setting up a sandbox OPNsense VM so I can test your config file.

[WaveZero]

@EasyG0ing1

@WaveZero I also wanted to mention that your IP Pool seems to have been assigned the entire subnet. While this is not necessarily a problem - because we assume that a static reservation won't be handed out from the pool ... it's generally best practice to define your pool outside of the addresses that you set aside for static reservation. The intent there is to guarantee no overlapping assignments as well as for management clarity etc.

And you have a lot of statics, you might consider dropping your subnet mask to a 23 bit mask which would make your network have these available IP addresses: 192.168.0.1 - 192.168.1.254 with the network address being 192.168.0.0 and the broadcast being 192.168.1.255 With a setup like that, you could define your DHCP pool as being 192.168.1.100 - 192.168.1.254 Then just do your static maps from x.x.0.1 to x.x.1.99 which gives you roughly 350-ish IP addresses while another 150 remain available for dynamic assignments.

Might be too much, but head room is always desireable with DHCP.

Agreed about going to a /23 mask, this network was a drop in replacement of my old Asus router network and I didn't want to change the network around too much. I will set up the IP pools as per your advice and will consider moving to a /23 network in the near future.

I have been relatively lazy in setting up a proper network for my home environment. I work day in and day out with enterprise networks, DC networks and carrier grade service provider networks (MPLS VPNs, SR-MPLS, etc) and I honestly can't be bothered managing a full blown home network.

@WaveZero - Did you create a new config.xml before running the migration and importing the new config?

I have generated a new config as per your advice in an earlier comment.

@WaveZero Im setting up a sandbox OPNsense VM so I can test your config file.

Thank you so much for going to such effort in assisting me, I appreciate your help and support thus far! If you have a Paypal address or something, I will donate some $$ to at least cover for a coffee or beer or something.

[EasyG0ing1]

@WaveZero

I have been relatively lazy in setting up a proper network for my home environment. I work day in and day out with enterprise networks, DC networks and carrier grade service provider networks (MPLS VPNs, SR-MPLS, etc) and I honestly can't be bothered managing a full blown home network.

You wouldn't happen to be hiring wherever you work, would you? I got over 20 years Network Engineering under my belt and about 6 years Java development. bout 10 years ago I started a managed services business but I was solo ... and doing very well until I got sick and almost died ... had to be on an IV at home for a full year 19 hours a day ... lost clients ... then my wife of 20 years decided she had to go and the rest is history. But currently, I'm in need of work. And I'm quite good at this stuff as it sounds like you are too. MPLS is a fun protocol ☺

[WaveZero]

@WaveZero

I have been relatively lazy in setting up a proper network for my home environment. I work day in and day out with enterprise networks, DC networks and carrier grade service provider networks (MPLS VPNs, SR-MPLS, etc) and I honestly can't be bothered managing a full blown home network.

You wouldn't happen to be hiring wherever you work, would you? I got over 20 years Network Engineering under my belt and about 6 years Java development. bout 10 years ago I started a managed services business but I was solo ... and doing very well until I got sick and almost died ... had to be on an IV at home for a full year 19 hours a day ... lost clients ... then my wife of 20 years decided she had to go and the rest is history. But currently, I'm in need of work. And I'm quite good at this stuff as it sounds like you are too. MPLS is a fun protocol ☺

So sorry to hear that you went through such hard times, I hope that you're in a better place now and your health is better. Yes, we are always looking for talent where I work, though you would have to move to Australia if you want to work here (we work with government and they don't allow offshore/overseas workers for compliance reasons), happy to recommend you if you wish to move here. We do a lot of scripting as well, I have extensive experience in Ansible and doing network automation using the Ansible framework with YAML, Jinja2 and Python. We are still using Ansible AWX but moving over to Ansible Tower in the near future. Most of the organisation's provisioning scripts were developed by me and it integrates nicely with Netbox to pull and build configs. Though I have moved on from doing low level technical work, I work mostly in presales doing solution design, writing proposals, working on major tenders, being the technical lead in major critical projects, etc.

[Passaita]

Hello, I have performed xml conversion in Linux Mint and also missing IPs and MAC addresses in KeaDHCP settings.

[EasyG0ing1]

@WaveZero

So sorry to hear that you went through such hard times, I hope that you're in a better place now and your health is better. Yes, we are always looking for talent where I work, though you would have to move to Australia if you want to work here (we work with government and they don't allow offshore/overseas workers for compliance reasons), happy to recommend you if you wish to move here. We do a lot of scripting as well, I have extensive experience in Ansible and doing network automation using the Ansible framework with YAML, Jinja2 and Python. We are still using Ansible AWX but moving over to Ansible Tower in the near future. Most of the organisation's provisioning scripts were developed by me and it integrates nicely with Netbox to pull and build configs. Though I have moved on from doing low level technical work, I work mostly in presales doing solution design, writing proposals, working on major tenders, being the technical lead in major critical projects, etc.

Health is great. When I was in the hospital for about 10 days, there were three doctors working with me. My main doctor, and two GIs that he was consulting with. Both GIs wanted to remove my entire lower intestine where I would have to live with a colostomy bag for the rest of my life. But THANK GOD my doctor - a hard working, soft spoken man from India said he wanted to try something different... his theory was to give the colon rest by not eating solid foods for almost a year then give me steroids and a bunch of other drugs that had fun side effects ... and see if the colon won't heal itself... was his idea... and his idea worked. SO, I lost my business, yes, but I kept my colon ... a worthy trade off if you ask me.

Moving to Austrailia sounds intriguing (I've played an authentic Didgeridoo before ☺ )... not sure I'm at that point yet. It would have to be a heck of a deal I think to re-locate. I'm looking for remote work ... I live in a rural area without much high end industry out here so there isn't much locally for my experience level and skills ... and covid made working remotely - acceptable so I'm trying to do that.

[EasyG0ing1]

Hello, I have performed xml conversion in Linux Mint and also missing IPs and MAC addresses in KeaDHCP settings.

I'm looking at it trying to figure it out.. Not sure why the IPs aren't coming in but Im sure it's something obscure or maybe even obvious ... just not obvious yet. :-)

[EasyG0ing1]

@WaveZero

So this is an interesting development. Even though the IP addresses and MAC addresses don't show up in the interface after an import, if you then take a new backup, they do show up in the config file which means that the IP addys and macs are in fact part of the config they just aren't showing up in the UI for some reason.

[WaveZero]

@WaveZero

So sorry to hear that you went through such hard times, I hope that you're in a better place now and your health is better. Yes, we are always looking for talent where I work, though you would have to move to Australia if you want to work here (we work with government and they don't allow offshore/overseas workers for compliance reasons), happy to recommend you if you wish to move here. We do a lot of scripting as well, I have extensive experience in Ansible and doing network automation using the Ansible framework with YAML, Jinja2 and Python. We are still using Ansible AWX but moving over to Ansible Tower in the near future. Most of the organisation's provisioning scripts were developed by me and it integrates nicely with Netbox to pull and build configs. Though I have moved on from doing low level technical work, I work mostly in presales doing solution design, writing proposals, working on major tenders, being the technical lead in major critical projects, etc.

Health is great. When I was in the hospital for about 10 days, there were three doctors working with me. My main doctor, and two GIs that he was consulting with. Both GIs wanted to remove my entire lower intestine where I would have to live with a colostomy bag for the rest of my life. But THANK GOD my doctor - a hard working, soft spoken man from India said he wanted to try something different... his theory was to give the colon rest by not eating solid foods for almost a year then give me steroids and a bunch of other drugs that had fun side effects ... and see if the colon won't heal itself... was his idea... and his idea worked. SO, I lost my business, yes, but I kept my colon ... a worthy trade off if you ask me.

Moving to Austrailia sounds intriguing (I've played an authentic Didgeridoo before ☺ )... not sure I'm at that point yet. It would have to be a heck of a deal I think to re-locate. I'm looking for remote work ... I live in a rural area without much high end industry out here so there isn't much locally for my experience level and skills ... and covid made working remotely - acceptable so I'm trying to do that.

I'm glad to hear that you're well now and that your doctor managed to save your colon.

RE: Moving to Australia, well if you're ever interested, feel free to ping me via email. I'm relatively well respected in my organisation and I do have a good network in the industry, so I'm sure I can help you find something where I live if you want to make that jump.

@WaveZero

So this is an interesting development. Even though the IP addresses and MAC addresses don't show up in the interface after an import, if you then take a new backup, they do show up in the config file which means that the IP addys and macs are in fact part of the config they just aren't showing up in the UI for some reason.

That is an interesting development, I have yet to reboot my OPNsense yet, maybe the IP addresses and MAC addresses will show up...

[Passaita]

@WaveZero

So this is an interesting development. Even though the IP addresses and MAC addresses don't show up in the interface after an import, if you then take a new backup, they do show up in the config file which means that the IP addys and macs are in fact part of the config they just aren't showing up in the UI for some reason.

That's correct, I can see those values in the config but not in the GUI. Can't reboot firewall midday (family if working from home).

[Passaita]

Can confirm that IPs and MAC addresses are missing in GUI after reboot.

[yfhyou]

I have the same issue using linux. I found that if I load the migrated file, and then add a new reservation manually through the GUI, the backup file loses all the previously shown MAC and IP addresses. Comparing files I see that it changed <ip_Address> to <ip_address>. Same capitalization fix for hw_Address.

I think you have a typo in the script :)

[yfhyou]

I can confirm that after changing the <ip_Address> ... </ip_Address> to <ip_address> ... </ip_address> (same for hw_Address -> hw_address) in my new_config.xml file, the import works as expected and all values are shown in the GUI. Didn't quite understand the code enough to write a PR.

[EasyG0ing1]

@WaveZero @Passaita @yfhyou Thank you all for your help in this. The release 2.1.2 fixes the problem