windmill-labs / windmill

Open-source developer platform to power your entire infra and turn scripts into webhooks, workflows and UIs. Fastest workflow engine (13x vs Airflow). Open-source alternative to Retool and Temporal.
https://windmill.dev
Other
10.78k stars 518 forks source link

feature: Any plans to support DB-triggered events? #2684

Open caryfitzhugh opened 12 months ago

caryfitzhugh commented 12 months ago

Looking at windmill, and a idea I had was to enable workflows to 'enrich' data as it is inserted into the DB.

Granted, I could put together a web-hook and have my PG database do some shenanigans to get that posted. Or I could trigger in some application code, but in the cases when there are multiple apps hitting a single DB, sometimes it might be nice to let the DB be the 'source of truth triggering)

To be able to run something like:

INSERT INTO windmill_hosts (creds) VALUES ('......'); -- Or something more secret, dunno how that part would work to config it
CREATE EXTENSION 'windmill';

CREATE FUNCTION enrich_daterz() RETURNS trigger AS $$
    BEGIN
        --  some function which gets the trigger event to windmill
        windmill_trigger('new_daterz_event', NEW.*);
        RETURN NULL;
    END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER enrich_daterz_trigger AFTER INSERT OR UPDATE ON daterz
    FOR EACH ROW EXECUTE FUNCTION enrich_daterz();

Just an idea I was playing with in my head.

slig commented 12 months ago

I believe that this can help you achieve that https://github.com/windmill-labs/windmill-adapters/tree/main/postgres-listen

rubenfiszel commented 12 months ago

@slig Tiago is absolutely right but we can probably improve the system in 2 ways:

Is the database shared with windmill ? Otherwise what do you see the major con of having your syntax translate to sending a webhook?

rubenfiszel commented 11 months ago

We now support this under the form of persistent scripts, more details coming soon but they are available in: Runtime -> Persistent Scripts

Those scripts are meant to run for a duration such as 24h (and exit on userland after that time) and listen to events. They are automatically restarted upon exiting.