network link between k8s nodegroup and pgsql via terraform

[k11h-de]

Current Provider Version

6.1.3

Use-cases

We would like to define a managed k8s cluster nodegroup which has a working network link to a managed postgresl Cluster in one terraform file.

Attempted Solutions

Right now I am using the attached file, but this is fundamentally not working from my perspective. Setting up a dedicated (user-managed) DHCP server only for this purpose is not an option.

Proposal

Implement an attribute in the ionoscloud_lan data source which returns the IP subnet the Ionos dhcp server will assign to clients in that LAN object. This attributes value could then be referenced in the pgsql resource -> connection -> cidr argument.

Terraform File

This is the terraform file we're using right now:

terraform {
  required_providers {
    ionoscloud = {
      source              = "ionos-cloud/ionoscloud"
      version             = "6.1.3"
    }
  }
}

variable "stack_name" {
  description = "a unique identifier for resource naming"  
  type        = string
  nullable    = false
  default     = "poc-cl2"
}

variable "nodegroup" {
  type = object({
    asg_min           = number
    asg_max           = number
    cores_count       = number
    ram_size          = number
    storage_size      = number 
    storage_type      = string  
  })  
  default = {      
    asg_min           = 1      
    asg_max           = 2      
    cores_count       = 2
    ram_size          = 4096
    storage_size      = 100  
    storage_type      = "SSD"  
  }
}

resource "ionoscloud_datacenter" "datacenter" {
  name                    = "${var.stack_name}_dc1"
  location                = "de/txl"
  description             = "proof of concept dc"
}

resource "ionoscloud_ipblock" "k8s_ingress_ip" {
  name                    = "${var.stack_name}_k8s_ingress_ips"
  location                = ionoscloud_datacenter.datacenter.location
  size                    = 1
}

resource "ionoscloud_lan" "lan1" {
  datacenter_id           = ionoscloud_datacenter.datacenter.id
  public                  = false
}

resource "ionoscloud_k8s_cluster" "k8s_cluster" {
  name                    = "${var.stack_name}_k8s_cluster"
  k8s_version             = "1.21.9"
  maintenance_window {
    day_of_the_week       = "Sunday"
    time                  = "22:30:00Z"
  }
}

resource "ionoscloud_k8s_node_pool" "k8s_cluster_nodegroup1" {
  name                    = "${var.stack_name}_k8s_cluster_ng1"
  k8s_version             = "1.21.9"
  auto_scaling {
    min_node_count        = var.nodegroup.asg_min
    max_node_count        = var.nodegroup.asg_max
  }

  lans {
    id                    = ionoscloud_lan.lan1.id
    dhcp                  = true
    routes {
      network             = "10.7.22.0/24"
      gateway_ip          = "10.7.22.17"
    } 
  }

  maintenance_window {
    day_of_the_week       = "Sunday"
    time                  = "22:30:00Z"
  }
  datacenter_id           = ionoscloud_datacenter.datacenter.id
  k8s_cluster_id          = ionoscloud_k8s_cluster.k8s_cluster.id
  cpu_family              = ionoscloud_datacenter.datacenter.cpu_architecture[0].cpu_family
  availability_zone       = "AUTO"
  storage_type            = var.nodegroup.storage_type
  node_count              = 1
  cores_count             = var.nodegroup.cores_count
  ram_size                = var.nodegroup.ram_size
  storage_size            = var.nodegroup.storage_size
  labels = {
    psp                   = "1234-123"
    team                  = "team-rocket"
  }
  annotations = {
    foo                   = "bar"
  }
}

resource "ionoscloud_pg_cluster" "pg_cluster_1" {
  postgres_version        = 13
  instances               = 2
  cores                   = 2
  ram                     = 2048
  storage_size            = 2048
  storage_type            = "SSD"
  connections   {
    datacenter_id         =  ionoscloud_datacenter.datacenter.id 
    lan_id                =  ionoscloud_lan.lan1.id 
    cidr                  =  "10.7.22.100/24"
  }
  location                = ionoscloud_datacenter.datacenter.location
  display_name            = "poc_bru1_pg_c1"
  maintenance_window {
    day_of_the_week       = "Sunday"
    time                  = "22:45:00"
  }
  credentials {
    username              = "user"
    password              = "insecure"
  }
  synchronization_mode    = "SYNCHRONOUS"
}

[iblindu]

Hi @k11h-de,

Thank you opening this issue! Unfortunately it is not possible for the moment to take the IP subnet directly from a ionoscloud_lan data source, since to find the subnet we have to look at the NIC configuration. Thus, here is an workaround:

• add in your plan a server in a private LAN
• then use one of the ips of the nic for taking the IP subnet and use it further for the cidr

Here is an example that I tested in this regard:

resource "ionoscloud_server" "test_dbaas_cluster" {
  name = "server"
  datacenter_id = ionoscloud_datacenter.test_dbaas_cluster.id
  cores = 2
  ram = 2048
  availability_zone = "ZONE_1"
  cpu_family = "INTEL_SKYLAKE"
  image_name ="Debian-10-cloud-init.qcow2"
  image_password = "K3tTj8G14a3EgKyNeeiYsasad"
  volume {
    name = "volume_test"
    size = 6
    disk_type = "SSD Standard"
  }
  nic {
    lan = ionoscloud_lan.test_dbaas_cluster.id
    name = "test_dbaas_cluster"
    dhcp = true
  }
}

locals {
 prefix = format("%s/%s", ionoscloud_server.test_dbaas_cluster.nic[0].ips[0], "24")
 database_ip = cidrhost(local.prefix, 1)
 database_ip_cidr = format("%s/%s", local.database_ip, "24")
}

resource "ionoscloud_pg_cluster" "test_dbaas_cluster" {
  postgres_version   = 12
  instances          = 1
  cores              = 4
  ram                = 2048
  storage_size       = 2048
  storage_type       = "HDD"
  connections  {
    datacenter_id   =  ionoscloud_datacenter.test_dbaas_cluster.id 
    lan_id          =  ionoscloud_lan.test_dbaas_cluster.id 
    cidr            =  local.database_ip_cidr
  }
  location = ionoscloud_datacenter.test_dbaas_cluster.location
  display_name = "PostgreSQL_cluster"
  credentials {
    username = "username"
    password = "password"
  }
  maintenance_window {
    day_of_the_week       = "Sunday"
    time                  = "09:00:00"
  }
  synchronization_mode    = "ASYNCHRONOUS"
}

P.S. We will also add all these details in the documentation. Thank you a lot for pointing this out!

[k11h-de]

Hi @IuliaBlindu , thanks a lot for your reply and the suggested workaround. I really like your idea and was wondering if there is a chance to get the value from the gateway_ip of the k8s nodegroup object in terraform.

That would require following change:

Now:

• If the attribute gateway_ip is not defined explicitly in terraform the value is returned as "gateway_ip" = tostring(null) in the datasource

Then:

• If the attribute gateway_ip is not defined explicitly in terraform the value is returned as "gateway_ip" = "10.7.22.1" This value would be needed to configure the nodes NIC anyways. One could use your snippet from above the construct the string "10.7.22.1/24" which one can then use in the pgsql object in terraform.

What do you think about this suggestion?

Thanks a million Karsten

[iblindu]

Hi @k11h-de,

I am not sure I understand the suggestion, so please correct me if I am wrong. Do you mean, when there is not a gateway_ip provided in the nodepool, to have a default value, hard coded by us?

[k11h-de]

Sorry @IuliaBlindu , I was not precise.

My understanding was that after creation terraform returns values like the ionoscloud_server.test_dbaas_cluster.nic[0].ips[0] in your example. So not hardcoded, but my understanding was that the CIDR of the node group is assigned when the k8s node group is created. Of course, one cannot return NIC details for a node group object as this is changing when nodes come and go, but I thought we can go with the gateway_ip as this should be in the same network segment.

[cristiGuranIonos]

We talked to the team that handle the nodepools, we discussed several options, but for now the only workaround is the one that @IuliaBlindu suggested above.

[Syndlex]

For People searching for a "Workaround" for Kubernetes. The Support send me this Modul. https://github.com/maboehm/terraform-ionoscloud-kube-lan-ip

It just takes the id of a Node from the Nodepool and gets the Server instance behind it. Now you can get the Nic from this Server.