rancher / terraform-provider-rancher2

Terraform Rancher2 provider
https://www.terraform.io/docs/providers/rancher2/
Mozilla Public License 2.0
253 stars 219 forks source link

Add note linking to AWS docs #1247

Closed mallardduck closed 4 months ago

mallardduck commented 9 months ago

Problem

Some customers come into Infrastructure as Code because they are Rancher users, rather than because they are embracing IaC out right. As such, some users don't realize they should user other Terraform providers than just Rancher. This gives the impression that if the Rancher TFP doesn't do a thing then it's a missing feature or bug.

Solution

We can provide users with context clues of when to user other terraform providers as necessary. In this case we are linking to the relevant AWS terraform data-sources. By telling users to fetch image IDs this way we are implicitly telling them to use other Terraform providers.

Testing

N/A; docs change.

Engineering Testing

Manual Testing

N/A; docs change.

Automated Testing

N/A; docs change.

QA Testing Considerations

N/A; docs change.

Regressions Considerations

N/A; docs change.

bennysp commented 9 months ago

Hi - I have helped work through this with SUSE vendor. I don't think this is all there is to this one unfortunately.

https://github.com/rancher/terraform-provider-rancher2/issues/927

The above issue/error is what we see when simply going from no launch template to a launch template in Rancher provider; It breaks because we are not providing an AMI, but that is not even enough to get it to work. AMI ID is not enough on it's own. You also need to then bootstrap it in order to get it to work too.

You have to do something like this in order to get it to work with the Rancher provided setup and a launch_template....


# get data for aws_ssm_parameters_by_path for var.kubernetes_version
# https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id.html
data "aws_ssm_parameter" "ami_imageid" {
  name = "/aws/service/eks/optimized-ami/${var.kubernetes_version}/amazon-linux-2/recommended/image_id"
}

# bootstrap the EKS nodes since we have to provide an image id as a workaround
# https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id.html
data "template_file" "ng_user_data" {
  template = <<-EOF
              #!/bin/bash
              set -o xtrace
              /etc/eks/bootstrap.sh ${var.eks_ds_name}
              EOF
}

resource "aws_launch_template" "rancher_ds_worker" {
  for_each = zipmap(var.node_groups[*].name, var.node_groups)

  name_prefix = "${var.eks_ds_name}-${each.value.name}-launchtemp"

  # 
  # When specifying launch template, do NOT specify these in here and keep in node group; limitation of AWS
  # https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html
  # DO NOT USE HERE: iam_instance_profile
  # 

  instance_type = each.value.node_instance_type
  image_id = data.aws_ssm_parameter.ami_imageid.value  # must provide a workaround to avoid a Rancher bug

  block_device_mappings {
    device_name = "/dev/xvda"
    ebs {
      volume_size = each.value.disk_size
      volume_type = "gp3"
      iops                  = 3000
      throughput            = 125
      encrypted             = true
      delete_on_termination = true
    }
  }
  update_default_version = true
  key_name = var.ec2_ssh_key
  vpc_security_group_ids = [var.eks_node_group_sg_id]

  user_data = base64encode(data.template_file.ng_user_data.rendered)
}

resource "rancher2_cluster" "rancher-cluster" {
  name = var.eks_ds_name
  description = "Terraform EKS downstream cluster"

  eks_config_v2 {
    cloud_credential_id = var.cloud_credential
    region = var.aws_region
    kubernetes_version = var.kubernetes_version

    subnets = var.subnet_ids
    logging_types = var.eks_logging_types

    security_groups = [var.eks_cluster_sg_id]

    dynamic "node_groups" {
      for_each = var.node_groups
      content {
        name            = "${var.eks_ds_name}-${node_groups.value["name"]}"
        subnets         = node_groups.value["subnet_ids"]
        node_role       = var.ng_iam_role

        # 
        # When specifying launch template, do NOT specify these in node groups; limitation of AWS
        # https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html
        # DO NOT USE HERE: ec2_ssh_key, instance_type, disk_size, image_id, user_data
        # 

        launch_template {
          id = resource.aws_launch_template.rancher_ds_worker[node_groups.value["name"]].id
          version = resource.aws_launch_template.rancher_ds_worker[node_groups.value["name"]].latest_version
          name = resource.aws_launch_template.rancher_ds_worker[node_groups.value["name"]].name
        }

        desired_size  = node_groups.value["des_dataplane_nodes"]
        max_size      = node_groups.value["max_dataplane_nodes"]
        min_size      = node_groups.value["min_dataplane_nodes"]
    }
  }
`
mjura commented 4 months ago

looks good, I have restarted CI