search

databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
454 stars 392 forks source link

[ISSUE] `make test` fails since 1.10.x when `DEFAULT` profile is defined in the `~/.databrickscfg` #2149

Open alexott opened 1 year ago

alexott commented 1 year ago

Expected Behavior

make test works without problems even when DEFAULT profile is defined

Actual Behavior

make test fails with errors since version 1.10.0, and it looks like it's trying to execute against the actual workspace - looks like a behavior change due Go SDK migration

log is below

Steps to Reproduce

  1. Add the DEFAULT profile
  2. Execute make test

Terraform and provider versions

provider: 1.10+

logs

✓ Filling vendor folder with library code ...
✓ Linting source code with https://staticcheck.io/ ...
✓ Running tests ...
∅  .
✓  aws (1.269s)
=== RUN   TestDatabricksClientConfigure_Nothing
2023/03/23 18:04:26 [TRACE] Loading config via environment
2023/03/23 18:04:26 [TRACE] Loading config via config-file
2023/03/23 18:04:26 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:26 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:26 [INFO] Auth is: pat
    client_test.go:30: 
            Error Trace:    /.../terraform-provider-databricks/common/client_test.go:30
                                        /.../terraform-provider-databricks/common/client_test.go:37
            Error:          Expected value not to be nil.
            Test:           TestDatabricksClientConfigure_Nothing
            Messages:       expected to have error: default auth: cannot configure default credentials
--- FAIL: TestDatabricksClientConfigure_Nothing (0.00s)
✖  common (1.365s)
✓  internal/acceptance (1.964s)
✓  libraries (1.117s)
✓  mlflow (917ms)
✓  commands (5.719s)
✓  jobs (3.784s)
✓  access (6.58s)
✓  policies (608ms)
✓  permissions (2.857s)
✓  catalog (8.277s)
✓  pools (635ms)
✓  pipelines (1.899s)
=== RUN   TestConfig_NoParams
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: 
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:94
            Error:          Expected value not to be nil.
            Test:           TestConfig_NoParams
            Messages:       Expected to have default auth: cannot configure default credentials error
--- FAIL: TestConfig_NoParams (0.01s)
=== RUN   TestConfig_HostEnv
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: host
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:103
            Error:          Expected value not to be nil.
            Test:           TestConfig_HostEnv
            Messages:       Expected to have default auth: cannot configure default credentials error
--- FAIL: TestConfig_HostEnv (0.01s)
=== RUN   TestConfig_ConfigFile
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: 
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:241
            Error:          Expected value not to be nil.
            Test:           TestConfig_ConfigFile
            Messages:       Expected to have default auth: cannot configure default credentials error
--- FAIL: TestConfig_ConfigFile (0.00s)
✖  provider (802ms)
✓  qa (642ms)
✓  clusters (10.672s)
✓  secrets (1.43s)
✓  serving (606ms)
=== RUN   TestWorkspaceTokenWrongAuthCornerCase
2023/03/23 18:04:35 [TRACE] Loading config via environment
2023/03/23 18:04:35 [TRACE] Loading config via config-file
2023/03/23 18:04:35 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:35 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:35 [TRACE] Loading config via environment
2023/03/23 18:04:35 [TRACE] Loading config via config-file
2023/03/23 18:04:35 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] POST /api/2.0/token/create
> {
>   "comment": "test",
>   "lifetime_seconds": 3600
> }
< HTTP/2.0 200 OK
< {
<   "token_info": {
<     "comment": "test",
<     "creation_time": 1679591076151,
<     "expiry_time": 1679594676151,
<     "token_id": "4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d"
<   },
<   "token_value": "**REDACTED**"
< }
2023/03/23 18:04:36 [TRACE] set workspace_url "https://adb-33333170229861029.9.azuredatabricks.net"
2023/03/23 18:04:36 [TRACE] set token []interface {}{map[string]interface {}{"comment":"test", "lifetime_seconds":3600, "token_id":"4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d", "token_value":"dapic793c4e6a2f0089aef9918eadb137283-2"}}
    resource_mws_workspaces_test.go:1198: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1198
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       create
2023/03/23 18:04:36 [TRACE] Loading config via environment
2023/03/23 18:04:36 [TRACE] Loading config via config-file
2023/03/23 18:04:36 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] GET /api/2.0/token/list
< HTTP/2.0 200 OK
< {
<   "token_infos": [
<     {
<       "comment": "test",
<       "creation_time": 1679591076151,
<       "expiry_time": 1679594676151,
<       "token_id": "4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d"
<     },
<     {
<       "comment": "test",
<       "creation_time": 1679591036084,
<       "expiry_time": 1679594636084,
<       "token_id": "e721d7d401a950359808bd113ec973abd019aef75fe26d03fc152e8b15471e98"
<     },
<     {
<       "comment": "test",
<       "creation_time": 1679590728131,
<       "expiry_time": 1679594328131,
<       "token_id": "2692f7006a3a7463e4dc67abfc8d42be79ef36dd7d69d05ba51cc33b51ac7771"
<     },
<     {
<       "comment": "cli",
<       "creation_time": 1668421557456,
<       "expiry_time": 1683973557456,
<       "token_id": "183df26be45a8aaad62f67f62ea1a8819125243c9654e3cfae358dd3e471c17a"
<     },
<     {
<       "comment": "azdo",
<       "creation_time": 1665334366557,
<       "expiry_time": 1680886366557,
<       "token_id": "8a09a795d26874954223e8d6116331f1293ab84f1592a2208de005188ac2a766"
<     }
<   ]
< }
    resource_mws_workspaces_test.go:1199: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1199
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       ensure
2023/03/23 18:04:36 [TRACE] Loading config via environment
2023/03/23 18:04:36 [TRACE] Loading config via config-file
2023/03/23 18:04:36 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] POST /api/2.0/token/delete
> {
>   "token_id": "x"
> }
< HTTP/2.0 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "No valid token with ID x found."
< }
    resource_mws_workspaces_test.go:1200: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1200
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       remove
--- FAIL: TestWorkspaceTokenWrongAuthCornerCase (1.05s)
✓  repos (1.827s)
✓  sql/api (243ms)
✖  mws (7.402s)
✓  exporter (13.251s)
✓  scim (3.862s)
✓  tokens (1.058s)
✓  workspace (2.027s)
✓  sql (4.651s)
✓  storage (5.529s)

=== Failed
=== FAIL: common TestDatabricksClientConfigure_Nothing (0.00s)
2023/03/23 18:04:26 [TRACE] Loading config via environment
2023/03/23 18:04:26 [TRACE] Loading config via config-file
2023/03/23 18:04:26 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:26 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:26 [INFO] Auth is: pat
    client_test.go:30: 
            Error Trace:    /.../terraform-provider-databricks/common/client_test.go:30
                                        /.../terraform-provider-databricks/common/client_test.go:37
            Error:          Expected value not to be nil.
            Test:           TestDatabricksClientConfigure_Nothing
            Messages:       expected to have error: default auth: cannot configure default credentials

=== FAIL: mws TestWorkspaceTokenWrongAuthCornerCase (1.05s)
2023/03/23 18:04:35 [TRACE] Loading config via environment
2023/03/23 18:04:35 [TRACE] Loading config via config-file
2023/03/23 18:04:35 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:35 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:35 [TRACE] Loading config via environment
2023/03/23 18:04:35 [TRACE] Loading config via config-file
2023/03/23 18:04:35 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] POST /api/2.0/token/create
> {
>   "comment": "test",
>   "lifetime_seconds": 3600
> }
< HTTP/2.0 200 OK
< {
<   "token_info": {
<     "comment": "test",
<     "creation_time": 1679591076151,
<     "expiry_time": 1679594676151,
<     "token_id": "4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d"
<   },
<   "token_value": "**REDACTED**"
< }
2023/03/23 18:04:36 [TRACE] set workspace_url "https://adb-33333170229861029.9.azuredatabricks.net"
2023/03/23 18:04:36 [TRACE] set token []interface {}{map[string]interface {}{"comment":"test", "lifetime_seconds":3600, "token_id":"4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d", "token_value":"....-2"}}
    resource_mws_workspaces_test.go:1198: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1198
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       create
2023/03/23 18:04:36 [TRACE] Loading config via environment
2023/03/23 18:04:36 [TRACE] Loading config via config-file
2023/03/23 18:04:36 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] GET /api/2.0/token/list
< HTTP/2.0 200 OK
< {
<   "token_infos": [
<     {
<       "comment": "test",
<       "creation_time": 1679591076151,
<       "expiry_time": 1679594676151,
<       "token_id": "4b9064c2f97843801a3a1421cc4c16d0cdfea9a0965ee09baeb91b079927e31d"
<     },
<     {
<       "comment": "test",
<       "creation_time": 1679591036084,
<       "expiry_time": 1679594636084,
<       "token_id": "e721d7d401a950359808bd113ec973abd019aef75fe26d03fc152e8b15471e98"
<     },
<     {
<       "comment": "test",
<       "creation_time": 1679590728131,
<       "expiry_time": 1679594328131,
<       "token_id": "2692f7006a3a7463e4dc67abfc8d42be79ef36dd7d69d05ba51cc33b51ac7771"
<     },
<     {
<       "comment": "cli",
<       "creation_time": 1668421557456,
<       "expiry_time": 1683973557456,
<       "token_id": "183df26be45a8aaad62f67f62ea1a8819125243c9654e3cfae358dd3e471c17a"
<     },
<     {
<       "comment": "azdo",
<       "creation_time": 1665334366557,
<       "expiry_time": 1680886366557,
<       "token_id": "8a09a795d26874954223e8d6116331f1293ab84f1592a2208de005188ac2a766"
<     }
<   ]
< }
    resource_mws_workspaces_test.go:1199: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1199
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       ensure
2023/03/23 18:04:36 [TRACE] Loading config via environment
2023/03/23 18:04:36 [TRACE] Loading config via config-file
2023/03/23 18:04:36 [TRACE] Attempting to configure auth: pat
2023/03/23 18:04:36 [DEBUG] POST /api/2.0/token/delete
> {
>   "token_id": "x"
> }
< HTTP/2.0 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "No valid token with ID x found."
< }
    resource_mws_workspaces_test.go:1200: 
            Error Trace:    /.../terraform-provider-databricks/mws/resource_mws_workspaces_test.go:1200
            Error:          An error is expected but got nil.
            Test:           TestWorkspaceTokenWrongAuthCornerCase
            Messages:       remove

=== FAIL: provider TestConfig_NoParams (0.01s)
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: 
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:94
            Error:          Expected value not to be nil.
            Test:           TestConfig_NoParams
            Messages:       Expected to have default auth: cannot configure default credentials error

=== FAIL: provider TestConfig_HostEnv (0.01s)
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: host
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:103
            Error:          Expected value not to be nil.
            Test:           TestConfig_HostEnv
            Messages:       Expected to have default auth: cannot configure default credentials error

=== FAIL: provider TestConfig_ConfigFile (0.00s)
2023/03/23 18:04:34 [INFO] Explicit and implicit attributes: 
2023/03/23 18:04:34 [TRACE] Loading config via environment
2023/03/23 18:04:34 [TRACE] Loading config via config-file
2023/03/23 18:04:34 [DEBUG] Loading DEFAULT profile from /Users/user/.databrickscfg
2023/03/23 18:04:34 [TRACE] Attempting to configure auth: pat
    provider_test.go:77: 
            Error Trace:    /.../terraform-provider-databricks/provider/provider_test.go:77
                                        /.../terraform-provider-databricks/provider/provider_test.go:241
            Error:          Expected value not to be nil.
            Test:           TestConfig_ConfigFile
            Messages:       Expected to have default auth: cannot configure default credentials error

DONE 1270 tests, 89 skipped, 5 failures in 22.130s
make: *** [test] Error 1
mgyucht commented 1 year ago

Option: override env variable for .databrickscfg path to point to nowhere, even if you have a ~/.databrickscfg, it will be ignored.

alexott commented 1 year ago

it's not very friendly as I often use databricks cli from the same session to check results, etc. It would be best to fix on the tests level