[Bug]: Error, Autotune is not supported in t2/t3 instance types. even though its DISABLED

[coolapso]

Terraform Core Version

0.13.7

AWS Provider Version

4.59.0

Affected Resource(s)

• aws_opensearch_domain

Expected Behavior

Resource created

Actual Behavior

Resource to be created

Relevant Error/Panic Output Snippet

Error: creating OpenSearch domain: ValidationException: Autotune is not supported in t2/t3 instance types. Disable autotune or change your instance type.

  on ../../../terraform/vaas/opensearch.tf line 138, in resource "aws_opensearch_domain" "this":
 138: resource "aws_opensearch_domain" "this" {

Terraform Configuration Files

Main Resource configuration:

resource "aws_opensearch_domain" "this" {
  count = var.build_opensearch ? 1 : 0

  domain_name    = var.client
  engine_version = var.os_engine_version

  vpc_options {
    subnet_ids         = [data.aws_subnet.private[0].id]
    security_group_ids = [aws_security_group.opensearch[0].id]
  }

  cluster_config {
    instance_count = var.os_instance_count
    instance_type  = var.os_instance_type

    dedicated_master_enabled = var.os_dedicated_master_enabled
    dedicated_master_count   = var.os_dedicated_master_count
    dedicated_master_type    = var.os_dedicated_master_type
    zone_awareness_enabled   = var.os_zone_awareness_enabled

  }

  ebs_options {
    ebs_enabled = true
    volume_size = var.os_volume_size
  }

  domain_endpoint_options {
    enforce_https       = true
    tls_security_policy = var.os_tls_policy
  }

  advanced_security_options {
    enabled                        = true
    anonymous_auth_enabled         = false
    internal_user_database_enabled = true

    master_user_options {
      master_user_name     = aws_ssm_parameter.os_username[count.index].value
      master_user_password = aws_ssm_parameter.os_password[count.index].value
    }
  }

  auto_tune_options {
    desired_state       = var.os_autotune
    rollback_on_disable = "NO_ROLLBACK"
  }

  encrypt_at_rest {
    enabled = true
  }

  node_to_node_encryption {
    enabled = true
  }

  log_publishing_options {
    enabled                  = var.os_enable_logging
    cloudwatch_log_group_arn = aws_cloudwatch_log_group.opensearch[0].arn
    log_type                 = var.os_logtype
  }

  tags       = local.tags
  depends_on = [aws_ssm_parameter.os_username, aws_ssm_parameter.os_password, aws_cloudwatch_log_group.opensearch, aws_cloudwatch_log_resource_policy.opensearch]
}

Variables:

 variable "build_opensearch" {
  type        = bool
  description = "Build Opensearch backend resources"
  default     = true
}

variable "os_engine_version" {
  type        = string
  description = "Opensearch engine version"
  default     = "OpenSearch_1.3"
}

variable "os_instance_type" {
  type        = string
  description = "Opensearch instance type"
}

variable "os_instance_count" {
  type        = number
  description = "The Number of opensearch data instances"
}

variable "os_dedicated_master_enabled" {
  type        = bool
  description = "Enables opensearch dedicated master instances"
  default     = false
}

variable "os_dedicated_master_count" {
  type        = number
  description = "The Number of opensearch dedicated master instances"
  default     = 3
}

variable "os_dedicated_master_type" {
  type        = string
  description = "The type of opensearch dedicated master instances"
  default     = "c6g.large.search"
}

variable "os_zone_awareness_enabled" {
  type        = bool
  description = "Enables opensearch zone awareness"
  default     = false
}

variable "os_volume_size" {
  type        = number
  description = "The size of opensearch data volume"
}

variable "os_autotune" {
  type        = string
  description = "Enables opensearch autotune"
  default     = "DISABLED"
}

variable "os_tls_policy" {
  type        = string
  description = "opensearch tls policy to be used"
  default     = "Policy-Min-TLS-1-2-2019-07"
}

variable "os_enable_logging" {
  type        = bool
  description = "Enables opensearch logging to cloudwatch"
  default     = false
}

variable "os_logtype" {
  type        = string
  description = "Type of OpenSearch log. Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, AUDIT_LOGS."
  default     = "ES_APPLICATION_LOGS"
}

variable "os_cw_retention" {
  type        = number
  description = "The number of days to retain opensearch logs in cloudwatch."
  default     = 30
}

Steps to Reproduce

• Set instance type to t3.small.search
• Set instance count to 1
• set autotune to "DISABLED"

Debug Output

No response

Panic Output

No response

Important Factoids

Resource plan output:

# aws_opensearch_domain.this[0] will be created
  + resource "aws_opensearch_domain" "this" {
      + access_policies    = (known after apply)
      + advanced_options   = (known after apply)
      + arn                = (known after apply)
      + dashboard_endpoint = (known after apply)
      + domain_id          = (known after apply)
      + domain_name        = "REDACTED"
      + endpoint           = (known after apply)
      + engine_version     = "OpenSearch_1.3"
      + id                 = (known after apply)
      + kibana_endpoint    = (known after apply)
      + tags               = {
          + "Client"      = "REDACTED"
          + "Environment" = "REDACTED"
          + "Name"        = "REDACTED"
          + "System"      = "REDACTED"
        }
      + tags_all           = {
          + "Client"      = "REDACTED"
          + "Environment" = "REDACTED"
          + "Name"        = "REDACTED"
          + "System"      = "REDACTED"
        }

      + advanced_security_options {
          + anonymous_auth_enabled         = false
          + enabled                        = true
          + internal_user_database_enabled = true

          + master_user_options {
              + master_user_name     = "REDACTED"
              + master_user_password = (sensitive value)
            }
        }

      + auto_tune_options {
          + desired_state       = "DISABLED"
          + rollback_on_disable = "NO_ROLLBACK"

          + maintenance_schedule {
              + cron_expression_for_recurrence = (known after apply)
              + start_at                       = (known after apply)

              + duration {
                  + unit  = (known after apply)
                  + value = (known after apply)
                }
            }
        }

      + cluster_config {
          + dedicated_master_enabled = false
          + instance_count           = 1
          + instance_type            = "t3.small.search"
          + zone_awareness_enabled   = false

          + cold_storage_options {
              + enabled = (known after apply)
            }
        }

      + domain_endpoint_options {
          + custom_endpoint_enabled = false
          + enforce_https           = true
          + tls_security_policy     = "Policy-Min-TLS-1-2-2019-07"
        }

      + ebs_options {
          + ebs_enabled = true
          + iops        = (known after apply)
          + throughput  = (known after apply)
          + volume_size = 20
          + volume_type = (known after apply)
        }

      + encrypt_at_rest {
          + enabled    = true
          + kms_key_id = (known after apply)
        }

      + log_publishing_options {
          + cloudwatch_log_group_arn = "REDACTED"
          + enabled                  = false
          + log_type                 = "ES_APPLICATION_LOGS"
        }

      + node_to_node_encryption {
          + enabled = true
        }

      + vpc_options {
          + availability_zones = (known after apply)
          + security_group_ids = [
              + "sg-redacted",
            ]
          + subnet_ids         = [
              + "subnet-redacted",
            ]
          + vpc_id             = (known after apply)
        }
    }

  # aws_opensearch_domain_policy.this[0] will be created
  + resource "aws_opensearch_domain_policy" "this" {
      + access_policies = (known after apply)
      + domain_name     = "REDACTED"
      + id              = (known after apply)
    }

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[coolapso]

in the meantime implemented work around for this issue:

dynamic "auto_tune_options" {
    for_each = var.os_autotune == "ENABLED" ? [1] : []

    content {
      desired_state       = var.os_autotune
      rollback_on_disable = "NO_ROLLBACK"
    }
  }

[tmielke-vizio]

The above solution outlined by @4s3ti works until you attempt to downgrade an existing cluster and instance type which supports auto-tune and has it enabled. In which case, the same error will manifest when applying the terraform plan. The workaround there is to manually disable auto-tune in the AWS console before downgrading the instance type via tf.

[gh-siyengar]

I am stuck on this error too, the workaround did not work. so +1 Unable to use the Opensearch service.

[StuartBertram]

I've hit the error while trying to reconfigure an existing T3 instance that had apparently had autotune enabled when it shouldn't have (but didn't have the agents) due to a bug in their deployment process.

AWS Support gave me the same answer as @tmielke-vizio suggests - manually disable autotune through the web UI before making changes. It isn't ideal, but it did work.