dlt-hub / verified-sources

Contribute to dlt verified sources đŸ”¥
https://dlthub.com/docs/walkthroughs/add-a-verified-source
Apache License 2.0
50 stars 40 forks source link

REST API source warns since 0.4.8 about missing authentication #420

Closed willi-mueller closed 2 months ago

willi-mueller commented 2 months ago

dlt version

0.4.8

Source name

rest_api

Describe the problem

Since I upgraded to dlt v0.4.8 I get this nasty warning. With v0.4.7 it is not present:

/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field token on BearerTokenAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field api_key on APIKeyAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field username on HttpBasicAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field password on HttpBasicAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field access_token on OAuth2AuthBase. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field client_id on OAuthJWTAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field private_key on OAuthJWTAuth. None assumed. All fields in configspec must have default.
  warnings.warn(
/Users/vilasa/Library/Caches/pypoetry/virtualenvs/demos-C1MIco0Z-py3.11/lib/python3.11/site-packages/dlt/common/configuration/specs/base_configuration.py:155: UserWarning: Missing default value for field auth_endpoint on OAuthJWTAuth. None assumed. All fields in configspec must have default.
  warnings.warn(

Expected behavior

The rest api source should not throw a warning when no authentication is provided because some APIs do not require authentication.

Steps to reproduce

import dlt

from rest_api import rest_api_source

pokemon_config = {
    "client": {
        "base_url": "https://pokeapi.co/api/v2/",
    },

    "resources": [
        "berry",
    ],
}

pokemon_source = rest_api_source(pokemon_config)

pipeline = dlt.pipeline(
    pipeline_name="pokemon_pipeline",
    destination="duckdb",
    dataset_name="pokemon",
    progress="log",
)

load_info = pipeline.run(pokemon_source)
print(load_info)

How you are using the source?

I run this source in production.

Operating system

macOS

Runtime environment

Local

Python version

3.11.4

dlt destination

duckdb

Additional information

No response

willi-mueller commented 2 months ago

Thank you very much for the quick fix in https://github.com/dlt-hub/verified-sources/pull/421 Cheers!

willi-mueller commented 2 months ago

Thank you very much for the quick fix in https://github.com/dlt-hub/verified-sources/pull/421