aws_elasticache_replication_group error when restoring redis cluster from snapshot - Slots must be provided when restoring from snapshot ARNs with cluster mode enabled

lancehudson commented 6 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.7

provider.aws v1.31.0
provider.chef v0.1.0
provider.external v1.0.0
provider.null v1.0.0
provider.template v1.0.

Affected Resource(s)

aws_elasticache_replication_group

Terraform Configuration Files

resource "aws_elasticache_replication_group" "redis-ap-southeast-2" {
  provider = "aws.stage-ap-southeast-2"
  engine                        = "redis"
  engine_version                = "4.0.10"
  replication_group_id          = "reds"
  replication_group_description = "redis"
  node_type                     = "cache.t2.medium"
  port                          = 6379
  parameter_group_name          = "default.redis4.0.cluster.on"
  automatic_failover_enabled    = true
  auto_minor_version_upgrade    = true
  at_rest_encryption_enabled    = true
  transit_encryption_enabled    = false

  subnet_group_name = "${aws_elasticache_subnet_group.redis-ap-southeast-2.name}"

  security_group_ids = [
      "${aws_security_group.redis-ap-southeast-2.id}"
  ]

  snapshot_arns = [
    "arn:aws:s3:::temp-redis-ap-southeast-2/0.rdb",
    "arn:aws:s3:::temp-redis-ap-southeast-2/1.rdb",
    "arn:aws:s3:::temp-redis-ap-southeast-2/2.rdb"
  ]

  maintenance_window = "sun:05:00-sun:09:00"
  snapshot_window = "05:00-09:00"

  cluster_mode {
    replicas_per_node_group     = 1
    num_node_groups             = 3
  }
}

Debug Output

Expected Behavior

Created redis cluster from snapshot

Actual Behavior

Error: Error applying plan:

1 error(s) occurred:

* aws_elasticache_replication_group.redis-ap-southeast-2: 1 error(s) occurred:
* aws_elasticache_replication_group.redis-ap-southeast-2: Error creating Elasticache Replication Group: InvalidParameterCombination: Slots must be provided when restoring from snapshot ARNs with cluster mode enabled
    status code: 400, request id: 1177d4c9-9d15-11e8-b12c-6347cdd5feee

Steps to Reproduce

terraform apply

Important Factoids

References

geota commented 5 years ago

This is impacting us as well, is there a way to accomplish this in TF?

https://github.com/terraform-providers/terraform-provider-aws/issues/5577 shows a work around that involves boto, but wondering if its possible to set the node-group-configuration from within TF

nhudson commented 5 years ago

Any update or timeline when this could get done? We are needing it.

ghost commented 4 years ago

This is impacting us as well. any plans to fix this soon.

antonioned commented 4 years ago

We are impacted as well, any plans to add this?

gpetrovgeorgi commented 3 years ago

Has there any resolution already prepared for this ? We are also getting it.

ashuec90 commented 3 years ago

Same issue for me also. getting error while creating cluster_replication_group from snapshot_arns

kodelint commented 3 years ago

I tried to take stab at it kodelint/terraform-provider-aws Which basically adds following:

node_group_configuration {
  slots = var.slots                                                       // Required - True
  primary_availability_zone = var.primary_availability_zone              // Required - False
  replica_availability_zones = var.replica_availability_zones              // Required - False
  replica_count = var.replica_count                                      // Required - False
}

tfproviders commented 2 years ago

I have given it spin and was able to implement it

resource "aws_elasticache_replication_group" "restore" {
  replication_group_id          = "tf-redis-cluster"
  replication_group_description = "test description"
  node_type                     = "cache.t2.small"
  port                          = 6379
  parameter_group_name          = "default.redis3.2.cluster.on"
  automatic_failover_enabled    = true
  snapshot_arns                 = [ "arn:aws:s3:::s3-bucket/snapshot-0001.rdb", "arn:aws:s3:::s3-bucket/snapshot-0002.rdb"]

  node_group_configuration {
    slots = [ "0-5461", "5462-10922" ]
  }

  cluster_mode {
    replicas_per_node_group = 1
    num_node_groups         = 2
  }
}

node_group_configuration block added to enable creating Redis Cluster from snapshots, documented here

nikunjundhad commented 2 years ago

@kodelint @tfproviders If your solutions are working as expected can you guys please create pull requests to get this merged into main provider. As our integration depend on original provider and they are updating other areas frequently.

pjpelegris commented 2 years ago

The changes that @tfproviders implemented to fix this are forked here: https://github.com/hashicorp/terraform-provider-aws/compare/main...tfproviders:terraform-provider-aws:main (His full article is here: https://awstip.com/amazon-redis-cache-terraform-build-cluster-from-snapshots-not-supported-wait-what-770c3f94b57f)

Changes that @kodelint made are hearder to pin down since it appears the commit he linked to above was done at the same time as the entire repo was copied out (https://github.com/kodelint/terraform-provider-aws/commits/main)

However, based on the name of the @kodelint user being Satyajit Roy and the @tfproviders post being by a person with the exact same name - I'm guessing they the same person and so there's one existing solution to fix this?

Thanks

kodelint commented 2 years ago

Hello @nikunjundhad and @pjpelegris Thanks for looking at the fork. I implemented these as my previous job required these feature. I am not in much use with these as of now. However, I plan to distribute and maintain these changes out of my tfproviders. Which can be downloaded from tfproviders/aws

@nikunjundhad I think I have submitted the PR sometime back, however not sure if it merged or not. I can try to create a new one with latest version.

davidreis97 commented 1 year ago

@kodelint if you still have a working fix for this a PR (and a merge) would be greatly appreciated. Let me know if I can be of help, I don't mind updating your fork and creating the PR (as it is almost 20k commits behind now)

rumitjain commented 1 year ago

I have given it spin and was able to implement it

resource "aws_elasticache_replication_group" "restore" {
  replication_group_id          = "tf-redis-cluster"
  replication_group_description = "test description"
  node_type                     = "cache.t2.small"
  port                          = 6379
  parameter_group_name          = "default.redis3.2.cluster.on"
  automatic_failover_enabled    = true
  snapshot_arns                 = [ "arn:aws:s3:::s3-bucket/snapshot-0001.rdb", "arn:aws:s3:::s3-bucket/snapshot-0002.rdb"]

  node_group_configuration {
    slots = [ "0-5461", "5462-10922" ]
  }

  cluster_mode {
    replicas_per_node_group = 1
    num_node_groups         = 2
  }
}

node_group_configuration block added to enable creating Redis Cluster from snapshots, documented here

Tried implementing this but getting error: Blocks of type "node_group_configuration" are not expected here.

hashicorp / terraform-provider-aws