databricks / dbt-databricks

A dbt adapter for Databricks.
https://databricks.com
Apache License 2.0
214 stars 115 forks source link

OAuth redirect running on a remote environment #705

Open juliengagnet-bib opened 3 months ago

juliengagnet-bib commented 3 months ago

Describe the bug

We are hosting DBT in a remote development environment (such as dbt.akme.com) using code-server are a looking at leveraging the OAuth H2M authentication.

When running the dbt debug command we are getting the error :

cannot listen on dbt.akme.com

Where it should only try to listen to localhost.

Steps To Reproduce

In the profiles.yml the the oauth_redirect_url as such:

oauth_redirect_url: "https://dbt.akme.com/codeserver/proxy/8020/"

And when running dbt debug you should get the error.

Expected behavior

When setting up a redirect url on a different host than localhost, I would still expect dbt to listen to localhost.

System information

The output of dbt --version:

Core:
  - installed: 1.8.2
  - latest:    1.8.2 - Up to date!

Plugins:
  - spark:      1.8.0 - Up to date!
  - databricks: 1.8.1 - Up to date!

The operating system you're using:

The output of python --version:

Python 3.11.2
benc-db commented 3 months ago

@jackyhu-db do you have ideas on this one?

juliengagnet-bib commented 1 month ago

Further comment on this is issue. We are are deploying an IDE through the web using Code Server, as such the developer are not working on a localhost environment.

So we are overriding the oauth_redirect_url to point back to the url of the remote IDE. What we would expect is for the DBT Driver to still listen on localhost and to create forward rules on the remote IDE that would forward the traffic coming back to the redirect_url to the listening process.

The issue is here: https://github.com/databricks/databricks-sdk-py/blob/main/databricks/sdk/oauth.py#L285 (on line 285)