Add proxy support

[timwisbauer-contsec]

Description

Add a new optional parameter to the provider configuration to allow for setting a proxy. Using a proxy can be an easier method for connecting to clusters within a VPC.

Issues Resolved

Closes https://github.com/opensearch-project/terraform-provider-opensearch/issues/93

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[prudhvigodithi]

Thanks for your contribution @timwisbauer-contsec , can you please add some unit tests to your change?

[timwisbauer-contsec]

@prudhvigodithi thank you for the review. I've added some unit tests to validate the proxy URL can be configured.

[prudhvigodithi]

Hey @timwisbauer-contsec thanks for adding unit tests, can you please resolve the conflicts and generate the documentation for this change ? Please check https://github.com/hashicorp/terraform-plugin-docs. We should merge this PR soon. :) Thank you @bbarani

[timwisbauer-contsec]

@prudhvigodithi I have resolved the conflicts and generated documentation. Thanks for your attention and please let me know if there's anything else I can do to help.

[prudhvigodithi]

Hey @timwisbauer-contsec thanks again, on qq, so if user uses proxy url does he still need the AWS credentials? or the idea is to bypass the AWS credentials and directly connect to the proxy URL ? Thank you

[timwisbauer-contsec]

Hey @timwisbauer-contsec thanks again, on qq, so if user uses proxy url does he still need the AWS credentials? or the idea is to bypass the AWS credentials and directly connect to the proxy URL ? Thank you

Hey @prudhvigodithi the proxy URL is separate from any credentials. The provider still needs to authenticate to the OpenSearch instance whether they're connecting to AWS or another OpenSearch instance.

For example, in our environment using the forked version I have the provider configured like this

provider "opensearch" {
  url   = var.opensearch_provider_enabled ? "https://${data.aws_opensearch_domain.domain[0].endpoint}" : ""
  proxy = "socks5://${var.socks_proxy_host}:${var.socks_proxy_port}"
}

The provider still needs to authenticate to AWS. In my case it reads my environment variable AWS_PROFILE when planning locally or container credentials (outlined here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration) when deploying from an ECS container.

[prudhvigodithi]

Thanks @timwisbauer-contsec so correct me if I'm wrong the proxy setting is not just for socks5:// but also can work with any other frontend proxies like haproxy, nginx etc (even though they start with https://)?

Example

provider "opensearch" {
  url   = var.opensearch_provider_enabled ? "https://${data.aws_opensearch_domain.domain[0].endpoint}" : ""
  proxy = "https://${var.ha_proxy_host}:${var.ha_proxy_port}"
}

So does the above code work ?

[timwisbauer-contsec]

Thanks @timwisbauer-contsec so correct me if I'm wrong the proxy setting is not just for socks5:// but also can work with any other frontend proxies like haproxy, nginx etc (even though they start with https://)?

Example

provider "opensearch" {
  url   = var.opensearch_provider_enabled ? "https://${data.aws_opensearch_domain.domain[0].endpoint}" : ""
  proxy = "https://${var.ha_proxy_host}:${var.ha_proxy_port}"
}

So does the above code work ?

@prudhvigodithi haproxy and nginx are reverse proxies which are different than the forward proxy configured here. Cloudflare has a decent explanation of the difference here: https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/

That being said, a proxy URL can alternatively use HTTP or HTTPS as the protocol so your example of

proxy = "https://${var.ha_proxy_host}:${var.ha_proxy_port}"

may be valid depending on the network topology between the client running terraform and the OpenSearch instance.

[prudhvigodithi]

Thanks @timwisbauer-contsec, my point is irrespective of proxy or revery proxy as the code looks generic

provider "opensearch" {
  url   = var.opensearch_provider_enabled ? "https://${data.aws_opensearch_domain.domain[0].endpoint}" : ""
  proxy = "https:// or http://"
}

does this logic work for any proxy ?

[timwisbauer-contsec]

Thanks @timwisbauer-contsec, my point is irrespective of proxy or revery proxy as the code looks generic

provider "opensearch" {
  url   = var.opensearch_provider_enabled ? "https://${data.aws_opensearch_domain.domain[0].endpoint}" : ""
  proxy = "https:// or http://"
}

does this logic work for any proxy ?

@prudhvigodithi yes, it works for any forward proxy. Basically if the HTTP transport in Go supports it you can use it here.

[prudhvigodithi]

Thanks, just thinking aloud, what If a user configure a revere proxy proxy = "https:// or http://" trying to access the OpenSearch server via terraform, will this setting still work ? @timwisbauer-contsec

[timwisbauer-contsec]

Thanks, just thinking aloud, what If a user configure a revere proxy proxy = "https:// or http://" trying to access the OpenSearch server via terraform, will this setting still work ? @timwisbauer-contsec

@prudhvigodithi yeah this setting just needs to meet the URL requirements here: https://pkg.go.dev/net/http#ProxyFromEnvironment

As long as the URL you've provided for the proxy is capable of handling those requests then this setting will work.

I haven't used nginx as a forward proxy before, but it looks like it's possible to be configured that way: https://www.baeldung.com/nginx-forward-proxy

[prudhvigodithi]

Thanks @timwisbauer-contsec LGTM.