Support ProxyFromEnvironment for honouring the http/https proxy settings

[hkantare]

We have request from client team where they working to provision the HPCS service keys using Terraform.

We developed Terraform resource to use this SDK to support key management for both HPCS and key-protect service.

When client tried to use the terraform client for HPCS,
it resulted in an error
dial tcp: lookup api.us-south.hs-crypto.cloud.ibm.com: no such host
client has a proxy setup , and all connections have to go through Proxy.
They are setting the HTTP_PROXY & HTTPS_PROXY variable, but it appears the client is not honouring those.

From the client

I would like to confirm that this is the consistent behaviour when we use terraform.
Curl commands are working consistently for us from the same shell.

We have set the environment variable HTTP_PROXY & HTTPS_PROXY to point to the client proxy which allows these URLs.
I think the terraform implementation doesn’t consider the proxy setting.
I removed the proxy variable from my shell & tried to use the curl command which resulted in similar error.

$ export HTTP_PROXY=""
$ export HTTPS_PROXY=""
$ curl -X GET https://api.us-south.hs-crypto.cloud.ibm.com:9068/api/v2/keys -H "authorization: Bearer <TOKEN>" -H "bluemix-instance: 0913f457-87e2-402d-a0eb-1aa9eeb8e394" -H "accept: application/vnd.ibm.kms.key+json"
curl: (6) Could not resolve host: api.us-south.hs-crypto.cloud.ibm.com

$ export HTTP_PROXY="<client proxy>"
$ export HTTPS_PROXY="<client proxy>"
$ curl -X GET https://api.us-south.hs-crypto.cloud.ibm.com:9068/api/v2/keys  -H "authorization: Bearer ${TOKEN}" -H "bluemix-instance: 0913f457-87e2-402d-a0eb-1aa9eeb8e394" -H "accept: application/vnd.ibm.kms.key+json"
{"metadata":{"collectionType":"application/vnd.ibm.kms.key+json","collectionTotal":2},"resources":[{"type":"application/vnd.ibm.kms.key+json","id":"01bcfe89-9908-4df4-827b-c49bb1376dda","name":"Root-key-1","description":"Root-key-1","state":1,"extractable":false,"crn":"crn:v1:bluemix:public:hs-crypto:us-south:a/761f90d452e44d278a603bebbafe74a2:0913f457-87e2-402d-a0eb-1aa9eeb8e394:key:01bcfe89-9908-4df4-827b-c49bb1376dda","imported":false,"creationDate":"2020-07-15T09:45:09Z","createdBy":"iam-ServiceId-9d276995-2f1b-4ebe-abda-bf5f43dc87c0","algorithmType":"AES","algorithmMetadata":{"bitLength":"256","mode":"CBC_PAD"},"algorithmBitSize":256,"algorithmMode":"CBC_PAD","lastUpdateDate":"2020-07-15T09:45:09Z","keyVersion":{"id":"01bcfe89-9908-4df4-827b-c49bb1376dda","creationDate":"2020-07-15T09:45:09Z"},"dualAuthDelete":{"enabled":false}},{"type":"application/vnd.ibm.kms.key+json","id":"bf07d420-fb41-431e-877d-eb98c9c89f81","name":"test-bucket-en-key","state":1,"extractable":true,"crn":"crn:v1:bluemix:public:hs-crypto:us-south:a/761f90d452e44d278a603bebbafe74a2:0913f457-87e2-402d-a0eb-1aa9eeb8e394:key:bf07d420-fb41-431e-877d-eb98c9c89f81","imported":false,"creationDate":"2020-08-27T10:20:07Z","createdBy":"IBMid-550005XB6M","algorithmType":"AES","algorithmMetadata":{"bitLength":"256","mode":"CBC_PAD"},"algorithmBitSize":256,"algorithmMode":"CBC_PAD","lastUpdateDate":"2020-08-27T10:20:07Z","dualAuthDelete":{"enabled":false}}]}

And it started working when I added the proxy environment variables back .

[hkantare]

https://golang.org/pkg/net/http/#ProxyFromEnvironment

var DefaultTransport RoundTripper = &Transport{
    Proxy: ProxyFromEnvironment,
    DialContext: (&net.Dialer{
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
        DualStack: true,
    }).DialContext,
    ForceAttemptHTTP2:     true,
    MaxIdleConns:          100,
    IdleConnTimeout:       90 * time.Second,
    TLSHandshakeTimeout:   10 * time.Second,
    ExpectContinueTimeout: 1 * time.Second,
}

[mrodden]

The client ignores the proxies by design. That effectively can void the warranty on the secure connection between the client and the KMS services (HPCS or KeyProtect.)

It is possible for the user of the SDK to inject their own transport with a proxy setting if they want... you just have to specify the Transport attribute on the New() constructor for the client.

This is something the terraform could detect and allow if you want, but I am not going to change the "secure by default" decision of ignoring HTTP_ and HTTPS_PROXY. It's far to easy to hijack the keys to a kingdom with that if the user is not aware.