CustomWebhook block type not found on Automation action

robfreedy commented 6 months ago

First check

[X] I added a descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the Prefect documentation for this issue.
[X] I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

Send a Notification Automation Action with CustomWebhook block fails after being triggered because the Block type could not be found. The CustomWebhook block does not exist out of the box in Prefect Cloud, so the block had to be registered to the workspace.

Reproduction

Register the Custom Webhook block using the command and python code below
Create the block by running the python code below
Create an automation with any trigger (used flow run state as completed for this example)
Set the Automation Action to "Send a notification" and specify the created Custom Webhook Block for the block. Save the automation.
Trigger the automation.

Creating the block:

integration_key = "some_secret"
    CustomWebhookNotificationBlock(
        name="custom-notifier",
        url="https://a-url-like-pager-duty",
        json_data={
            "routing_key": "{{routing_key}}",
            "event_action": "trigger",
            "payload": "{{body}}"},
        secrets={"routing_key": integration_key,
                 "payload": None
                 }
    ).save("custom-webhook-test-block", overwrite=True)

Command to register the block

prefect block register --file test-block.py

Error

This was taken from the event payload of the Automation action Failed event (see complete payload in screenshot below).

"The notification block was invalid: KeyError(\"No class found for dispatch key 'custom-webhook' in registry for type 'Block'.\")"

Versions

Version:             2.16.6
API version:         0.8.4
Python version:      3.11.8
Git commit:          3fecd435
Built:               Sat, Mar 23, 2024 4:06 PM
OS/Arch:             darwin/arm64
Profile:             default
Server type:         cloud

Additional context

Automation Action Failed event payload:

zangell44 commented 6 months ago

@robfreedy at the moment only known system block types (like Webhook) can be used in Automation actions.

We don't have any way to instantiate CustomWebhookNotificationBlock server side, only metadata about the block definition.

Happy to chat through a feature request if you think there's value.

Seems like at minimum we could explain the above in our Automations action error message.

robfreedy commented 6 months ago

Gotcha, that makes sense. I could see this experience being confusing based on what is exposed in the Cloud UI just because there could be any number of custom blocks with the "notify" tag on them when setting the block for the "Send a Notifaction" action that would not be valid since there is no way to instantiate the CustomWebhookNotificationBlock or any custom block type.

I would maybe propose we either:

Have a set list of "notification" block types that can be set on the Automation Action for sending a notification.
Support any block type (including custom block types) in the automation action (larger feature request/discussion)

cc: @masonmenges

PrefectHQ / prefect