opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.31k stars 738 forks source link

Kea configuration of first entry gets overwritten by new entries #7268

Closed moogle19 closed 7 months ago

moogle19 commented 7 months ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

We have multiple subnets in Kea. When we add a new one without "Auto collect option data" the Gateway and DNS servers of the first entry are overwritten by the values used in the new subnet. There are also inconsistencies between the Kea API and the opnsense user interface (see Screenshots).

To Reproduce

Steps to reproduce the behavior:

  1. Create multiple subnets with manual gateway and dns server in Kea.
  2. Add a new subnet with manual gateway and dns server.
  3. Check the gateway and dns server for the first subnet in the list

Expected behavior The first entry should not be overwritten

Screenshots This is the interface configuration in the user interface (192.168.70.0/24 net look correct, 192.168.90.0/24 has a wrong gateway and dns from a subnet we added later):

Screenshot 2024-02-22 at 15 15 26 Screenshot 2024-02-22 at 15 15 14

This is what the API returns (192.168.90.0/24 net look correct, 192.168.70.0/24 has a wrong gateway and dns):

[
  {
    "arguments": {
      "Dhcp4": {
        "allocator": "iterative",
        "authoritative": false,
        "boot-file-name": "",
        "calculate-tee-times": false,
        "control-socket": {
          "socket-name": "/var/run/kea4-ctrl-socket",
          "socket-type": "unix"
        },
        "ddns-generated-prefix": "myhost",
        "ddns-override-client-update": false,
        "ddns-override-no-update": false,
        "ddns-qualifying-suffix": "",
        "ddns-replace-client-name": "never",
        "ddns-send-updates": true,
        "ddns-update-on-renew": false,
        "ddns-use-conflict-resolution": true,
        "decline-probation-period": 86400,
        "dhcp-ddns": {
          "enable-updates": false,
          "max-queue-size": 1024,
          "ncr-format": "JSON",
          "ncr-protocol": "UDP",
          "sender-ip": "0.0.0.0",
          "sender-port": 0,
          "server-ip": "127.0.0.1",
          "server-port": 53001
        },
        "dhcp-queue-control": {
          "capacity": 64,
          "enable-queue": false,
          "queue-type": "kea-ring4"
        },
        "dhcp4o6-port": 0,
        "early-global-reservations-lookup": false,
        "echo-client-id": true,
        "expired-leases-processing": {
          "flush-reclaimed-timer-wait-time": 25,
          "hold-reclaimed-time": 3600,
          "max-reclaim-leases": 100,
          "max-reclaim-time": 250,
          "reclaim-timer-wait-time": 10,
          "unwarned-reclaim-cycles": 5
        },
        "hooks-libraries": [
          {
            "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so",
            "parameters": {}
          }
        ],
        "host-reservation-identifiers": [
          "hw-address",
          "duid",
          "circuit-id",
          "client-id"
        ],
        "hostname-char-replacement": "",
        "hostname-char-set": "[^A-Za-z0-9.-]",
        "interfaces-config": {
          "interfaces": [
            "vlan0.100",
            "vlan0.120",
            "vlan0.50",
            "vlan0.70",
            "vlan0.90"
          ],
          "re-detect": true
        },
        "ip-reservations-unique": true,
        "lease-database": { "persist": true, "type": "memfile" },
        "loggers": [
          {
            "debuglevel": 0,
            "name": "kea-dhcp4",
            "output_options": [
              { "flush": true, "output": "syslog", "pattern": "" }
            ],
            "severity": "INFO"
          }
        ],
        "match-client-id": true,
        "multi-threading": {
          "enable-multi-threading": true,
          "packet-queue-size": 64,
          "thread-pool-size": 0
        },
        "next-server": "0.0.0.0",
        "option-data": [],
        "option-def": [],
        "parked-packet-limit": 256,
        "reservations-global": false,
        "reservations-in-subnet": true,
        "reservations-lookup-first": false,
        "reservations-out-of-pool": false,
        "sanity-checks": {
          "extended-info-checks": "fix",
          "lease-checks": "warn"
        },
        "server-hostname": "",
        "server-tag": "",
        "shared-networks": [],
        "statistic-default-sample-age": 0,
        "statistic-default-sample-count": 20,
        "store-extended-info": false,
        "subnet4": [
          {
            "4o6-interface": "",
            "4o6-interface-id": "",
            "4o6-subnet": "",
            "allocator": "iterative",
            "calculate-tee-times": false,
            "id": 1,
            "max-valid-lifetime": 4000,
            "min-valid-lifetime": 4000,
            "option-data": [
              {
                "always-send": false,
                "code": 6,
                "csv-format": true,
                "data": "192.168.103.249",
                "name": "domain-name-servers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 3,
                "csv-format": true,
                "data": "192.168.103.249",
                "name": "routers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 15,
                "csv-format": true,
                "data": "localdomain",
                "name": "domain-name",
                "never-send": false,
                "space": "dhcp4"
              }
            ],
            "pools": [
              { "option-data": [], "pool": "192.168.70.20-192.168.70.209" }
            ],
            "relay": { "ip-addresses": [] },
            "reservations": [
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xxxxxxxxxxxxxx",
                "ip-address": "192.168.70.1",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              }
            ],
            "store-extended-info": false,
            "subnet": "192.168.70.0/24",
            "t1-percent": 0.5,
            "t2-percent": 0.875,
            "valid-lifetime": 4000
          },
          {
            "4o6-interface": "",
            "4o6-interface-id": "",
            "4o6-subnet": "",
            "allocator": "iterative",
            "calculate-tee-times": false,
            "id": 2,
            "max-valid-lifetime": 4000,
            "min-valid-lifetime": 4000,
            "option-data": [
              {
                "always-send": false,
                "code": 6,
                "csv-format": true,
                "data": "192.168.90.249",
                "name": "domain-name-servers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 3,
                "csv-format": true,
                "data": "192.168.90.249",
                "name": "routers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 15,
                "csv-format": true,
                "data": "localdomain",
                "name": "domain-name",
                "never-send": false,
                "space": "dhcp4"
              }
            ],
            "pools": [
              { "option-data": [], "pool": "192.168.90.10-192.168.90.240" }
            ],
            "relay": { "ip-addresses": [] },
            "reservations": [],
            "store-extended-info": false,
            "subnet": "192.168.90.0/24",
            "t1-percent": 0.5,
            "t2-percent": 0.875,
            "valid-lifetime": 4000
          },
          {
            "4o6-interface": "",
            "4o6-interface-id": "",
            "4o6-subnet": "",
            "allocator": "iterative",
            "calculate-tee-times": false,
            "id": 3,
            "max-valid-lifetime": 4000,
            "min-valid-lifetime": 4000,
            "option-data": [
              {
                "always-send": false,
                "code": 6,
                "csv-format": true,
                "data": "192.168.120.249",
                "name": "domain-name-servers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 3,
                "csv-format": true,
                "data": "192.168.120.249",
                "name": "routers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 15,
                "csv-format": true,
                "data": "localdomain",
                "name": "domain-name",
                "never-send": false,
                "space": "dhcp4"
              }
            ],
            "pools": [
              { "option-data": [], "pool": "192.168.120.10-192.168.120.240" }
            ],
            "relay": { "ip-addresses": [] },
            "reservations": [
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.120.60",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              }
            ],
            "store-extended-info": false,
            "subnet": "192.168.120.0/24",
            "t1-percent": 0.5,
            "t2-percent": 0.875,
            "valid-lifetime": 4000
          },
          {
            "4o6-interface": "",
            "4o6-interface-id": "",
            "4o6-subnet": "",
            "allocator": "iterative",
            "calculate-tee-times": false,
            "id": 4,
            "max-valid-lifetime": 4000,
            "min-valid-lifetime": 4000,
            "option-data": [
              {
                "always-send": false,
                "code": 6,
                "csv-format": true,
                "data": "192.168.50.249",
                "name": "domain-name-servers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 3,
                "csv-format": true,
                "data": "192.168.50.249",
                "name": "routers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 15,
                "csv-format": true,
                "data": "localdomain",
                "name": "domain-name",
                "never-send": false,
                "space": "dhcp4"
              }
            ],
            "pools": [
              { "option-data": [], "pool": "192.168.50.20-192.168.50.100" }
            ],
            "relay": { "ip-addresses": [] },
            "reservations": [
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.6",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              },
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.5",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              },
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.25",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              },
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.24",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              },
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.23",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              },
              {
                "boot-file-name": "",
                "client-classes": [],
                "hostname": "",
                "hw-address": "xx:xx:xx:xx:xx:xx",
                "ip-address": "192.168.50.26",
                "next-server": "0.0.0.0",
                "option-data": [],
                "server-hostname": ""
              }
            ],
            "store-extended-info": false,
            "subnet": "192.168.50.0/24",
            "t1-percent": 0.5,
            "t2-percent": 0.875,
            "valid-lifetime": 4000
          },
          {
            "4o6-interface": "",
            "4o6-interface-id": "",
            "4o6-subnet": "",
            "allocator": "iterative",
            "calculate-tee-times": false,
            "id": 5,
            "max-valid-lifetime": 4000,
            "min-valid-lifetime": 4000,
            "option-data": [
              {
                "always-send": false,
                "code": 6,
                "csv-format": true,
                "data": "192.168.103.249",
                "name": "domain-name-servers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 3,
                "csv-format": true,
                "data": "192.168.103.249",
                "name": "routers",
                "never-send": false,
                "space": "dhcp4"
              },
              {
                "always-send": false,
                "code": 15,
                "csv-format": true,
                "data": "localdomain",
                "name": "domain-name",
                "never-send": false,
                "space": "dhcp4"
              }
            ],
            "pools": [
              { "option-data": [], "pool": "192.168.100.10-192.168.103.240" }
            ],
            "relay": { "ip-addresses": [] },
            "reservations": [],
            "store-extended-info": false,
            "subnet": "192.168.100.0/22",
            "t1-percent": 0.5,
            "t2-percent": 0.875,
            "valid-lifetime": 4000
          }
        ],
        "t1-percent": 0.5,
        "t2-percent": 0.875,
        "valid-lifetime": 4000
      },
      "hash": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    },
    "result": 0
  }
]

Relevant log files

If applicable, information from log files supporting your claim.

Additional context

Add any other context about the problem here.

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 24.1.2_1-amd64 FreeBSD 13.2-RELEASE-p10 OpenSSL 3.0.13 Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (8 cores, 8 threads)

AdSchellevis commented 7 months ago

best check the contents of /usr/local/etc/kea/kea-dhcp4.conf as that is what we generate.

moogle19 commented 7 months ago

@AdSchellevis Hmm, kea-dhcp4.conf matches the values in the web interface. Not sure why the API yields different values.

mnaiman commented 7 months ago

Hi have same issue, and when I edit any non-first entry and save, first entry get corrupted even in kea-dhcp4.conf

AdSchellevis commented 7 months ago

A kea-dhcp4.conf file combined with the kea configuration section (in config.xml) might help track possible issues....

moogle19 commented 7 months ago

@AdSchellevis config.xml after first subnet:

  <Kea>
      <ctrl_agent version="0.0.1">
        <general>
          <enabled>1</enabled>
          <http_host>127.0.0.1</http_host>
          <http_port>8000</http_port>
        </general>
      </ctrl_agent>
      <dhcp4 version="1.0.0">
        <general>
          <enabled>1</enabled>
          <interfaces>opt20</interfaces>
          <valid_lifetime>4000</valid_lifetime>
          <fwrules>0</fwrules>
        </general>
        <ha>
          <enabled>0</enabled>
          <this_server_name/>
        </ha>
        <subnets>
          <subnet4 uuid="b3b776e5-0cfc-4510-b80a-9210ec814f4c">
            <subnet>192.168.0.0/24</subnet>
            <option_data_autocollect>0</option_data_autocollect>
            <option_data>
              <domain_name_servers>192.168.0.1</domain_name_servers>
              <routers>192.168.0.1</routers>
              <domain_name/>
              <ntp_servers/>
              <tftp_server_name/>
              <boot_file_name/>
            </option_data>
            <pools>192.168.0.10-192.168.0.240</pools>
          </subnet4>
        </subnets>
        <reservations/>
        <ha_peers/>
      </dhcp4>
    </Kea>

kea-dhcp4.conf after first subnet:

{
    "Dhcp4": {
        "valid-lifetime": 4000,
        "interfaces-config": {
            "interfaces": ["vlan0.803"]
        },
        "lease-database": {
            "type": "memfile",
            "persist": true
        },
        "control-socket": {
            "socket-type": "unix",
            "socket-name": "/var/run/kea4-ctrl-socket"
        },
        "loggers": [
            {
                "name": "kea-dhcp4",
                "output_options": [
                    {
                        "output": "syslog"
                    }
                ],
                "severity": "INFO"
            }
        ],
        "subnet4": [
            {
                "id": 1,
                "subnet": "192.168.0.0/24",
                "option-data": [
                    {
                        "name": "domain-name-servers",
                        "data": "192.168.0.1"
                    },
                    {
                        "name": "routers",
                        "data": "192.168.0.1"
                    },
                    {
                        "name": "domain-name",
                        "data": "localdomain"
                    }
                ],
                "pools": [
                    { "pool": "192.168.0.10-192.168.0.240" }
                ],
                "reservations": [
                ]
            }
        ]
        ,"hooks-libraries": [
            {
                "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so",
                "parameters": { }
            },
        ]
    }
}

config.xml after second subnet added:

  <Kea>
      <ctrl_agent version="0.0.1">
        <general>
          <enabled>1</enabled>
          <http_host>127.0.0.1</http_host>
          <http_port>8000</http_port>
        </general>
      </ctrl_agent>
      <dhcp4 version="1.0.0">
        <general>
          <enabled>1</enabled>
          <interfaces>opt20</interfaces>
          <valid_lifetime>4000</valid_lifetime>
          <fwrules>0</fwrules>
        </general>
        <ha>
          <enabled>0</enabled>
          <this_server_name/>
        </ha>
        <subnets>
          <subnet4 uuid="b3b776e5-0cfc-4510-b80a-9210ec814f4c">
            <subnet>192.168.0.0/24</subnet>
            <option_data_autocollect>0</option_data_autocollect>
            <option_data>
              <domain_name_servers>192.168.178.1</domain_name_servers>
              <routers>192.168.178.1</routers>
              <domain_name/>
              <ntp_servers/>
              <tftp_server_name/>
              <boot_file_name/>
            </option_data>
            <pools>192.168.0.10-192.168.0.240</pools>
          </subnet4>
          <subnet4 uuid="07b8fa5b-7754-4c59-8694-a75df52f468c">
            <subnet>192.168.178.0/24</subnet>
            <option_data_autocollect>0</option_data_autocollect>
            <option_data>
              <domain_name_servers>192.168.178.1</domain_name_servers>
              <routers>192.168.178.1</routers>
              <domain_name/>
              <ntp_servers/>
              <tftp_server_name/>
              <boot_file_name/>
            </option_data>
            <pools>192.168.178.10-192.168.178.240</pools>
          </subnet4>
        </subnets>
        <reservations/>
        <ha_peers/>
      </dhcp4>
    </Kea>

kea-dhcp4.conf after second subnet:

{
    "Dhcp4": {
        "valid-lifetime": 4000,
        "interfaces-config": {
            "interfaces": ["vlan0.803"]
        },
        "lease-database": {
            "type": "memfile",
            "persist": true
        },
        "control-socket": {
            "socket-type": "unix",
            "socket-name": "/var/run/kea4-ctrl-socket"
        },
        "loggers": [
            {
                "name": "kea-dhcp4",
                "output_options": [
                    {
                        "output": "syslog"
                    }
                ],
                "severity": "INFO"
            }
        ],
        "subnet4": [
            {
                "id": 1,
                "subnet": "192.168.0.0/24",
                "option-data": [
                    {
                        "name": "domain-name-servers",
                        "data": "192.168.178.1"
                    },
                    {
                        "name": "routers",
                        "data": "192.168.178.1"
                    },
                    {
                        "name": "domain-name",
                        "data": "localdomain"
                    }
                ],
                "pools": [
                    { "pool": "192.168.0.10-192.168.0.240" }
                ],
                "reservations": [
                ]
            },
            {
                "id": 2,
                "subnet": "192.168.178.0/24",
                "option-data": [
                    {
                        "name": "domain-name-servers",
                        "data": "192.168.178.1"
                    },
                    {
                        "name": "routers",
                        "data": "192.168.178.1"
                    },
                    {
                        "name": "domain-name",
                        "data": "localdomain"
                    }
                ],
                "pools": [
                    { "pool": "192.168.178.10-192.168.178.240" }
                ],
                "reservations": [
                ]
            }
        ]
        ,"hooks-libraries": [
            {
                "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so",
                "parameters": { }
            },
        ]
    }
}
AdSchellevis commented 7 months ago

@moogle19 from my perspective this looks good, same as specified in https://kea.readthedocs.io/en/latest/arm/dhcp4-srv.html or am I missing something?

moogle19 commented 7 months ago

@AdSchellevis The files are valid, the problem is that domain-name-servers and routers in the option-data values of the first entry are overwritten by the second entry.

I configured 192.168.0.1 as router/dns for the first subnet but after adding the second subnet with 192.168.178.1 the first one gets the router/dns specified in the 192.168.178.0 subnet.

AdSchellevis commented 7 months ago

@moogle19 after save it changes to the first?

AdSchellevis commented 7 months ago

I can't reproduce that on my end, auto collect is the only thing that mangles with these settings when set, the rest operates the same as any other form.

https://github.com/opnsense/core/blob/efc9b09c769b1d073a197517a25d9ea20667e234/src/opnsense/mvc/app/models/OPNsense/Kea/KeaDhcpv4.php#L48-L65

mnaiman commented 7 months ago

Yes, after save of second, first is corrupted. It happens when auto collect is off (could that be an issue)?

AdSchellevis commented 7 months ago

and what does the configuration history show? (System: Configuration: History)

AdSchellevis commented 7 months ago

starting from scratch I seem to be able to reproduce this, not sure why this happens yet though.

AdSchellevis commented 7 months ago

@moogle19 @mmaction does this issue on your end only happen when adding a new item or also when editing an existing one (and 2 are already there)?

moogle19 commented 7 months ago

@AdSchellevis It only happens when adding a new entry

AdSchellevis commented 7 months ago

@moogle19 ok, thanks, then I know where to look.

AdSchellevis commented 7 months ago

Should be fixed with https://github.com/opnsense/core/commit/8973f6efb14bec010e7f929ae748d8e06795e8c9 , which can be installed using:

opnsense-patch 8973f6e

Kea is the only component that currently uses these nested structures inside the model, which is the reason this issue only shows itself here.