dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
https://dlthub.com/docs
Apache License 2.0
2.65k stars 175 forks source link

rest_api: Allow specifying custom session #1843

Closed willi-mueller closed 1 month ago

willi-mueller commented 1 month ago

Feature description

I want to pass into the RESTAPIConfig a custom session object because it will allow me to use OAuth 1.0 and OAuth 2.0 via requests-oauthlib.

Are you a dlt user?

None

Use case

I want to connect to Oracle NetSuite using OAuth 1.0

With the proposed patch the following code works:

from requests_oauthlib import OAuth1Session

import dlt
from dlt.sources.rest_api import rest_api_source, RESTAPIConfig
from dlt.common import pendulum

netsuite_oauth = OAuth1Session(
  client_key=dlt.secrets["sources.netsuite.client_id"],
  client_secret=dlt.secrets["sources.netsuite.client_secret"],
  resource_owner_key=dlt.secrets["sources.netsuite.token_id"],
  resource_owner_secret=dlt.secrets["sources.netsuite.token_secret"],
  signature_method="HMAC-SHA256",
  realm=dlt.secrets["sources.netsuite.realm"],
  timestamp=str(pendulum.now().int_timestamp),
)

config: RESTAPIConfig = {
    "client": {
        "base_url": f"https://{dlt.secrets["sources.netsuite.realm"]}.restlets.api.netsuite.com/app/site/hosting/",
        "headers": {"content_type": "application/json"},
        "session": netsuite_oauth,  # This needs a modification in the rest_api_source
    },

    "resource_defaults": {
      "write_disposition": "replace",
      "endpoint": {
        "method": "POST",
      }
    },

  resources: [
    # ...  
  ]
}

Proposed solution

The RESTClient's constructor has a session parameter. Expose this also to the RESTAPIConfig.

Related issues

No response

rudolfix commented 1 month ago

@willi-mueller if you take a ticket it is helpful to assign yourself and move it to "planned/in progress" in the board