[Feature]: hcloud_server_network: assign IP from supplied subnet

[apricote]

What whould you like to see?

When assigning a server to a network through the hcloud_server_network resource, it is not possible to authoritavely choose the subnet that will be used. From out current docs:

subnet_id - (Optional, string) ID of the sub-network which should be added to the Server. Required if network_id is not set. Note: if the ip property is missing, the Server is currently added to the last created subnet.

This happens because the API does not support "attaching to a subnet", it only supports attaching to a Network with an optional IP. This IP would be used to determine the subnet the server is put into.

If a users wants the server to be "put into the subnet", they would need to explicitly calculate an IP from that subnet and pass that to the resource. While this is easy to do in simple setups, it can add dev cost in larger setups.

I am not really sure how we can implement this, besides generating random IPs from the subnet and trying to attach the server as this IP, but this always has the potential to conflict because that IP is already in use. Getting all assigned IPs and just assigning the next one has an even larger potential for conflicts, because that same IP will be chosen by all hcloud_server_network resources that are evaluated at the same time.

[Rohmilchkaese]

Push for that one. That exactly is the issue I encountered.

This is the code I use, to assign to private IPs:

#Asign private IPs to Master
resource "hcloud_server_network" "private_subnet_master" {
  for_each  = local.master
  server_id = lookup(hcloud_server.master[each.key], "id")
  subnet_id = hcloud_network_subnet.private_subnet_master.id
}

#Asign private IPs to Worker
resource "hcloud_server_network" "private_subnet_worker" {
  for_each  = local.worker
  server_id = lookup(hcloud_server.worker[each.key], "id")
  subnet_id = hcloud_network_subnet.private_subnet_worker.id
}

Which in return results in:

This is, from my point of view, not working as expected.. I really don't mind that much if IPs are not "logical" assigned in a sequential manner. But I'd really appreciate if I could decide to which subnet a Servers gets assigned. I guess there is no point in having more than one subnet in the base private net, if you can't use it in a predictable manner.

[mysticaltech]

Yep, that's something that would make the lifecycle of kube-hetzner clusters easier to manage too. As right now we have to pre-compute the IPs for subnets (one for each nodepool).

The problem with the current approach we have, is that users cannot delete a nodepool once created (they can only scale its count to 0), otherwise it would mess-up all the previous subnets.

[mikhailbot]

I thought I got around this issue by assigning IP's, but I'm still getting multiple instances of servers being put into the wrong subnet (with wrong IP).

resource "hcloud_server_network" "server_network" {
  count = var.server_count

  server_id  = hcloud_server.server[count.index].id
  network_id = var.network_id
  ip         = cidrhost(var.subnet_cidr_block, count.index + 1)
}

I create a network and multiple subnets based on CIDR blocks, that same variable is then passed to the module creating the servers, along with a depends_on, yet it still takes a couple runs of apply for things to get the right IP address. The count.index + 1 works, but the first one ends up in the wrong subnet?

Terraform 1.4.6 hetznercloud/hcloud 1.39.0

[apricote]

Hey @mikhailbot,

the first IP in every subnet is reserved for the subnet gateway: https://docs.hetzner.com/cloud/networks/faq/#are-any-ip-addresses-reserved

The first IP address of your network IP range. For example, in 10.0.0.0/8, you cannot use 10.0.0.1.

You can try to assign count.index + 2 instead.

Hope I could help you. If you have further questions, please open a new issue :)

[mikhailbot]

Thanks for info @apricote!

Couple things though, first I figured out my IP assignment issue was a stray network block inside hcloud_server. Either do it in the network block or do it with hcloud_server_network, not half a config in one and half in another.

Regarding subnets yes I moved things up to start at 10.10.0.0/24, 10.20.0.0/24, etc. to avoid this issue. It seems within those subnets the .1 isn't reserved? Instances work fine there and have no networking issues (internally or going out via a NAT host with the coinciding network route).

[github-actions[bot]]

This issue has been marked as stale because it has not had recent activity. The bot will close the issue if no further action occurs.

[alexandernst]

I'll take the risk and I'll ask the obvious question that nobody has asked (or I couldn't find?). Can't you implement an endpoint to attach/detach resources to/from a subnet?

[apricote]

I have already opened a feature request with the responsible team when I created this issue. Added a +1 to it for you too.