Unable to create snapshot repository in AWS Opensearch

[ShahroZafar]

Using terraform version v1.3.0 with phillbaker provider version 2.0.4, getting the following error while creating snapshot repository in AWS Opensearch version 1.3 whereas the rest of the resource creation requests are working fine

 Error: elastic: Error 403 (Forbidden)

The provider configurations that I am using are these

provider "elasticsearch" {
  url         = "https://xxx.us-east-1.es.amazonaws.com"
  username    = "xxx"
  password    = "xxx"
  healthcheck = false
  sign_aws_requests = false
}

and the resource configurations are as follows

resource "elasticsearch_snapshot_repository" "repo" {
  name = "xxx"
  type = "s3"
  settings = {
    bucket    = "xxx"
    region    = "us-east-1"
    role_arn  = "arn:aws:iam::xxx:role/xxx"
  }
}

When I use custom python script using a particular user and IAM role, the snapshot repository is registered. However, using the same user and IAM role in terraform gives the above mentioned error.

With terraform provider debug logs enabled, getting this

2022-10-10T05:55:51.361Z [INFO]  provider: configuring client automatic mTLS
2022-10-10T05:55:51.372Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/phillbaker/elasticsearch/2.0.4/linux_amd64/terraform-provider-elasticsearch_v2.0.4 args=[.terraform/providers/registry.terraform.io/phillbaker/elasticsearch/2.0.4/linux_amd64/terraform-provider-elasticsearch_v2.0.4]
2022-10-10T05:55:51.373Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/phillbaker/elasticsearch/2.0.4/linux_amd64/terraform-provider-elasticsearch_v2.0.4 pid=195278
2022-10-10T05:55:51.373Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/phillbaker/elasticsearch/2.0.4/linux_amd64/terraform-provider-elasticsearch_v2.0.4
2022-10-10T05:55:51.382Z [INFO]  provider.terraform-provider-elasticsearch_v2.0.4: configuring server automatic mTLS: timestamp=2022-10-10T05:55:51.382Z
2022-10-10T05:55:51.408Z [DEBUG] provider.terraform-provider-elasticsearch_v2.0.4: plugin address: network=unix address=/tmp/plugin1874154396 timestamp=2022-10-10T05:55:51.408Z
2022-10-10T05:55:51.408Z [DEBUG] provider: using plugin: version=5
module.test.elasticsearch_snapshot_repository.repo: Creating...
2022-10-10T05:55:51.450Z [INFO]  provider.terraform-provider-elasticsearch_v2.0.4: 2022/10/10 05:55:51 [INFO] Pinging url to determine version https://xxx.us-east-1.es.amazonaws.com with timeout 5s: timestamp=2022-10-10T05:55:51.450Z
2022-10-10T05:55:51.485Z [INFO]  provider.terraform-provider-elasticsearch_v2.0.4: 2022/10/10 05:55:51 [INFO] ES version {Number:1.3.2 BuildFlavor: BuildType:tar BuildHash:unknown BuildDate:2022-06-17T17:51:48.230231Z BuildSnapshot:false LuceneVersion:8.10.1 MinimumWireCompatibilityVersion:6.8.0 MinimumIndexCompatibilityVersion:6.0.0-beta1}: timestamp=2022-10-10T05:55:51.485Z
╷
│ Error: elastic: Error 403 (Forbidden)
│ 
│   with module.test.elasticsearch_snapshot_repository.repo,
│   on ../modules/ism_env/main.tf line 8, in resource "elasticsearch_snapshot_repository" "repo":
│    8: resource "elasticsearch_snapshot_repository" "repo" {
│ 
╵
2022-10-10T05:55:51.533Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2022-10-10T05:55:51.535Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/phillbaker/elasticsearch/2.0.4/linux_amd64/terraform-provider-elasticsearch_v2.0.4 pid=195278
2022-10-10T05:55:51.535Z [DEBUG] provider: plugin exited

[phillbaker]

Hello, what steps have you taken so far to debug the issue? The error message is descriptive in this case: the provider does not have permission to access the cluster. You can also search this repository for similar issues in the past: https://github.com/phillbaker/terraform-provider-elasticsearch/issues?q=is%3Aissue+403

Please include the following information:

• from where are you running the provider?
• where is the elasticsearch cluster located?
• can you access the cluster from where the provider is being run with the same credentials using curl?

Note: Issues on this repository are for reporting bugs and feature requests for this provider, not providing support for unique environments. In order to investigate this, a reproducible case should be provided, if that can't be provided, we'll have to close this issue to focus on widely impacting issues.

[saharpure]

Hi @phillbaker - i am having the same issue getting forbidden error on creating snapshot repository - i had no issue adding ISM policy to the domain and mapping policy to indices but when I try adding snapshot repository I get forbidden error: Error: elastic: Error 403 (Forbidden) │ │ with elasticsearch_snapshot_repository.sample_snapshot_repository, │ on policy.tf line 2, in resource "elasticsearch_snapshot_repository" "sample_snapshot_repository": │ 2: resource "elasticsearch_snapshot_repository" "sample_snapshot_repository" { │

to answer questions you asked above - what do you mean "from where are you running the provider?" ES cluster is in AWS opensearch domain

[saharpure]

okay - after playing around with it, it turned out the url in elastcisearch config was not set correctly:

provider "elasticsearch" { url = "https://MY_ES_DOMAIN.com" } posting if someone else has similar issue