dlt-hub / verified-sources

Contribute to dlt verified sources 🔥
https://dlthub.com/docs/walkthroughs/add-a-verified-source
Apache License 2.0
48 stars 38 forks source link

rest_api: Extend `response_actions` to accept hook functions #505

Open burnash opened 1 week ago

burnash commented 1 week ago

Background

Currently, the response_actions configuration within the REST API source only supports basic string actions like "ignore".

{
    "path": "issues",
    "response_actions": [
        {"status_code": 404, "action": "ignore"},
        {"content": "Not found", "action": "ignore"},
        {"status_code": 200, "content": "some text", "action": "ignore"},
    ],
}

Proposal

Extend the response_actions configuration to:

  1. Accept Python function as "action" to allow for more complex actions:

def custom_action(response):
    # Custom logic here
    return response

# ...
{
    "path": "issues",
    "response_actions": [
        {"status_code": 404, "action": custom_action},
    ],
}
  1. Accept a function as an action for all response statuses and content:

def custom_action(response):
    # Custom logic here
    return response

# ...
{
    "path": "issues",
    "response_actions": [
        custom_action,
    ],
}
  1. Accept multiple functions as an action :

def custom_action_1(response):
    # Custom logic here
    return response

def custom_action_2(response):
    # Custom logic here
    return response

def custom_action_3(response):
    # Custom logic here
    return response

# ...
{
    "path": "issues",
    "response_actions": [
        custom_action_1,
        custom_action_2
    ],
}

{
    "path": "issues",
    "response_actions": [
        custom_action_1,
        {"status_code": 200, "action": custom_action_2 }
    ],
}

{
    "path": "issues",
    "response_actions": [
        custom_action_1,
        {"status_code": 200, "action": [custom_action_2, custom_action_3] }
    ],
}

Task