Bump python sdk version

[eric-wang-1990]

Upgrade python sdk version from 0.17.0 to 0.36.0 Create DatabricksCredentialManager to managed credentials, under the hood use Config from databricks.sdk.core to manage the auth part which comes from the Python SDK. For dbt we support 4 different auth mode:

• token : When user explicitly passed in a token=xxxx
• external_browser: When user pass in auth_type=oauth with clientid=xxx and no client secret
• oauth-m2m: When user pass in clientid=xxx and clientsecret=yyy
• azure-client-secret: When user pass in azure_client_id=xxx and azure_client_secret=yyy and your host is an azure host

For oauth-m2m and azure-client-secret we set a preferred auth sequence based on if the clientSecret starts with "dose", so we can reduce the number of false trying as much as possible.

Remove the set/get sharedPassword part, since the python sdk already handles that, but with one defect which I already raise a PR against: https://github.com/databricks/databricks-sdk-py/pull/823

With this change one thing that will no longer work is if customer are using non-databricks OAuth client for U2M case it will not work, since the python sdk does not support modify redirectUrl/scopes.

Description

Checklist

• [x] I have run this code in development and it appears to resolve the stated issue
• [x] This PR includes tests, or tests are not required/relevant for this PR
• [ ] I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.

[jackyhu-db]

Thanks @eric-wang-1990 for updating this. Can your rename the PR title to make it more readable?

[benc-db]

Hi Eric, since the last time we were looking at this PR, 1.9 has become the target in main. So please rebase and target main with PR.

[benc-db]

Looks like something is off in the python path, with those E2E tests failing.