[FEATURE] Add our own DHCP Server

[alexlovelltroy]

OpenCHAMI needs a way to run dhcp directly as well as feed an external DHCP server with configuration data. This issue is confined to running a simple dhcp server directly within the docker compose environment.

Options Available

There are several credible, embeddable DHCP servers to consider

• ISC DHCP is a common choice even though it is EOL. There are plenty of libraries that help to generate configs for it. It is deprecated with kea as the successor.
• kea is the successor to ISC DHCP and is included with CSM. It has modern features like database backends, a rest interface, and modularity
• *dnsmasq is an all-purpose DHCP and DNS server that is commonly included in appliances. In addition to DHCP, it serves DNS , BOOTP, and PXE within the same small footprint. It's most recent release is a year old and it doesn't support external integrations.
• coredhcp is a relatively new project that can be used as a standalone dhcp server and can be extended with plugins. It also supports being embedded as a library with the same configuration and plugins.

Assessment

My recommendation is that we embed coredhcp using the file plugin for now and add a roadmap item to create a dedicated ochami plugin that interacts with the inventory system and some form of IPAM, (either within the coredhcp plugin or within ochami)

[njones-lanl]

Given the nature of a dhcp request (being a broadcast communication), I'd wonder if we could have separate handlers for giving a mapped to inventory IP address and handling a yet to be mapped IP address.

That would leave us having a very simple DHCP service, and having a standalone service to catch unmapped entries and prep them for SMD or similar.

[alexlovelltroy]

I like that idea. Just to be sure I understand, you're suggesting that for any broadcast domain we create two DHCP pools. A pool for known addresses and a different pool for unknown addresses that need to be discovered or probed in some way.

[njones-lanl]

I'd argue for the second pool way may not even want to use a full DHCP pool, depending on what we want to do with un-inventoried nodes.

It could just be a microservice that catches the info, passes it to SMD (or a service that we can use to integrate new nodes into SMD), and shuts the node off via BSS or something like that. I also think that avoiding the node itself running a script avoids a cyclical dependency between a booting node and the inventory system.

[njones-lanl]

I suspect the main catch here may be reloading dhcp entries when the config changes for dnsmasq, it accepts a SIGHUP at least: https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html#lbAG

[alexlovelltroy]

Once we have a container, I'm pretty sure I can get fast config reloads to work. docker kill --signal=SIGHUP should do the trick.

[synackd]

create a dedicated ochami plugin that interacts with the inventory system and some form of IPAM, (either within the coredhcp plugin or within ochami)

I think this is a feasible idea. Since SMD is managing all of the node information, it makes sense to have it be the single source of truth for DHCP configuration as well. Long term, I think having a plugin for coredhcp that, e.g., provides a RESTful API for updating/deleting/adding DHCP configuration that SMD can push to when changes occur is desirable. That way, we don't have to reimplement DHCP and we are still following the microservice architecture. At current, I am not sure if this would be possible with coredhcp, but the concept of being able to configure DHCP via an API without having to reimplement DHCP seems ideal here.

I'm aware that kea already provides a RESTful API for DHCP, but I've heard from colleagues administering it in CSM that it's not great to work with.

[alexlovelltroy]

Assigning @njones-lanl who has been working on a dnsmasq version for SI that will be good enough for now. We can revisit the plugin-based approach in the future.