[BUG]

[shalom-alloy]

What is the bug?

Getting errors when trying to use aws_assume_role_arn. Error: NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors

If I add directly the role of terraform(aws_profile) to the security_managers opensearch role, it works like a charm - but when trying to assume one of the roles that are already in there - I get that error message.

How can one reproduce the bug?

Try using the role assumption for any opensearch change and the above error shows up.

What is the expected behavior?

Role will get assumed and it's permissions applied when access opensearch.

What is your host/environment?

AWS opensearch domain - Opensearch 2.5 cluster with fine grained access control applied.

Do you have any additional context?

AWS environment, opensearch domain version 2.5 with fine grained control enabled, trying to add role mapping vis the opensearch provider(version 1.0.0). It works as long as I don't try to assume a role(using aws_profile), assuming a role throws the error mentioned above.

The relevant provider config in terraform looks like that(removing sensitive data): provider "opensearch" { url = "https://${aws_elasticsearch_domain.this.endpoint}" aws_region = data.aws_region.current.name sign_aws_requests = true healthcheck = false opensearch_version = "OpenSearch_2.5" aws_assume_role_arn = "arn:aws:iam::xxxxxxxxxxxxxx:role/yyyyyyyyyyyyyyy" }

[prudhvigodithi]

[Untriage] There is a recent PR merged with this topic https://github.com/opensearch-project/terraform-provider-opensearch/pull/54, @9rnt can you suggest how to make this work with assume role. Adding @phillbaker Thank you

[reneeckstein]

Hi @prudhvigodithi: I can confirm the reported issue. I just tested version 2.0.0-beta.1 of the provider and got the same errors without changing the provider configuration used with v1.0.0.

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│   For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│
│   with opensearch_roles_mapping.kibana_user,
│   on opensearch-settings.tf line 11, in resource "opensearch_roles_mapping" "kibana_user":
│   11: resource "opensearch_roles_mapping" "kibana_user" {

Provider config

provider "opensearch" {
  url                         = "https://${aws_opensearch_domain.es.endpoint}"
  aws_region                  = local.region
  aws_assume_role_arn         = var.assume_role
  healthcheck                 = true
}

I checked the updated provider usage documentation and tested the described default for Assume role configuration and et voilà it works :)

provider "opensearch" {
  url                         = "https://${aws_opensearch_domain.es.endpoint}"
  aws_region                  = local.region
  aws_assume_role_arn         = var.assume_role
  aws_assume_role_external_id = "SecretID"
  healthcheck                 = true
}

I tried different values for aws_assume_role_external_id and it always works. Looks like a bug, we don't use External ID with this IAM role, so I should not need to set something here in order to prevent failing at all.

[prudhvigodithi]

Hey thanks @reneeckstein, its good at least not blocking and allowing to move forward, can you please contribute to fix the bug with your suggestion ? Thank you

[estebane-frb]

Hm I'm getting this error even when I include aws_assume_role_external_id

[prudhvigodithi]

Closing this issue and PR related to issue is merged https://github.com/opensearch-project/terraform-provider-opensearch/pull/76. @shalom-alloy @estebane-frb @reneeckstein please feel free to comment/re-open if required. Adding @phillbaker @bbarani @peterzhuamazon Thank you

[viktord]

@shalom-alloy @estebane-frb @prudhvigodithi I'm getting the same error using provider v2.0.0. Would you pls test on your side and if you experience the same issue, reopen the bug. Thanks!

[estebane-frb]

Just tested it again with v2.0.0 and still experiencing the same issue. I don't have access to reopen this bug. @shalom-alloy are you or @prudhvigodithi able to reopen it?

[prudhvigodithi]

Reopening the bug, adding @ekirmayer, can you please add some insights with the PR merged https://github.com/opensearch-project/terraform-provider-opensearch/pull/76 Thank you

[ekirmayer]

@viktord @estebane-frb @prudhvigodithi the change included two parts:

1. You dont have to pass the external ID unless you use it. If you dont pass a value, it will set it to empty string and will fail. Now, if you dont need it, dont pass it.
2. If you set the assume role, and not the profile, it will fail as the profile is an empty string "". I've changed it to instead to use the default profile and not an empty string.

I've tested the new provider (2.0.0) on a 2.7 cluster. Here is my code. As you can see, i dont set the external id as i dont need it.

provider "opensearch" {
  url                 = "https://${var.sub_domain_name}.${var.domain_name}"
  aws_region          = var.aws_region
  sign_aws_requests   = true
  aws_assume_role_arn = "arn:aws:iam::11111111111111:role/some-role-with-access"
  healthcheck         = false
  aws_profile         = var.aws_active_profile
}

Let me know if this doesnt work and i will try to run it on a 2.5 cluster

[estebane-frb]

Hi @ekirmayer I just tried the above and I still have the same error. I wonder - should the awsprofile be required? In my case I am using a role assumed by my pipeline server so there won't be a profile. Or for example in cases where I use AWS* environment variables. I tried setting it to "default" but I still got the same error.

[ekirmayer]

@estebane-frb the way the provider is defined at the moment, as i understand it, is that you have to pass a profile. This is why i've set it to at-least take the default. I do believe that this is not a must and can be change. Regarding your error, please try to assume the role with your active profile/user/session and run a test call like aws s3 ls. This is just to make sure there are no errors in the assume role. The trust policy need to be correct (i've made few mistakes there). If this works, we can think on all the paths that might be needed. Sadly, i still dont have automatic pipeline so didnt failed on it yet

[viktord]

Same here, I've tested it without aws_profile

[moritzzimmer]

Same here, I guess setting a profile or assuming the default one should not be necessary

[estebane-frb]

@estebane-frb the way the provider is defined at the moment, as i understand it, is that you have to pass a profile. This is why i've set it to at-least take the default. I do believe that this is not a must and can be change. Regarding your error, please try to assume the role with your active profile/user/session and run a test call like aws s3 ls. This is just to make sure there are no errors in the assume role. The trust policy need to be correct (i've made few mistakes there). If this works, we can think on all the paths that might be needed. Sadly, i still dont have automatic pipeline so didnt failed on it yet

Hi @ekirmayer finished testing the assume role again with your asks but unfortunately we are still getting the same error. I also have a side question - does the provider currently support non-standard partitions like China and GovCloud OpenSearch partitions? (aws-cn, aws-us-gov) respectively.

[ekirmayer]

@estebane-frb can you send me a gist of code example for our provider and maybe a sample for Opensearch Cluster? I've tested this on both 2.5 & 2.7 but all on the general Zones, non in cn or gov. Are you working only on these regions? Did you try on these regions or also other? Are you using a public api or a vpc? Is this with reverse proxy or from within the VPC?

[massimob76]

@estebane-frb the way the provider is defined at the moment, as i understand it, is that you have to pass a profile. This is why i've set it to at-least take the default. I do believe that this is not a must and can be change.

I think the default profile should be the problem. So if you don't specify a profile it assumes that you are using the 'default' profile. Specifying the profile will inhibit aws-sdk-go to use the default provider chain as the library will try to fetch the credentials only using the profile (and not use environment variables injected credentials for example or other mechanisms in the provider chain).

There is a small PR #87 that should fix it (sorry I didn't notice this thread here before)

[michaeldross2]

Is this fixed in release 2.1.0?

[prudhvigodithi]

Since the PR was merged https://github.com/opensearch-project/terraform-provider-opensearch/pull/87 and released in 2.1.0, can we close this issue? or can you folks please confirm if the issue was resolved? Thanks @getsaurabh02