supabase / cli

Supabase CLI. Manage postgres migrations, run Supabase locally, deploy edge functions. Postgres backups. Generating types from your database schema.
https://supabase.com/docs/reference/cli/about
MIT License
1.02k stars 201 forks source link

Set up Webhooks in the config file #1190

Open NatoNathan opened 1 year ago

NatoNathan commented 1 year ago

At the moment you have to manually setup database webhooks. You also need to redo the setup when you reset the DB. This becomes a problem when collaborating with others and sharing projects. It make it harder to provide a single command to setup and configure a project locally or remote.

It would be nice to be able to use the config.toml file to define webhooks that are automatically setup when you start or reset a project. Maybe something like this?

[[db.webhooks]]
name = "my_webhook"
table= "auth.users"
operation = ["INSERT", "UPDATE", "DELETE"]
url = "http://localhost:3000/webhook"
params = { key = "value" }
headers = { key = "value" }

You can use SQL to make custom webhooks with triggers. but then you can't easily use environment variables for things like the url, params or headers.

sweatybridge commented 1 year ago

An alternative is to create web hooks from studio UI and save them as as a new migration.

supabase db diff --schema supabase_functions -f my_webhook

Would the above work for you?

EDIT: nvm, we actually need to save the insert statements not the web hooks schema.

NatoNathan commented 1 year ago

The important thing is to be able to set a webhook like this

[[db.webhooks]]
name = "my_webhook"
table= "auth.users"
operation = ["INSERT", "UPDATE", "DELETE"]
url = "env(API_URL)"
params = { Authorization = "env(JWT)" }
headers = { key = "value" }

where API_URL and JWT change between local, staging and prod. otherwise I would be happy to use SQL to define the trigger.

duyleekun commented 1 year ago

Would localhost works in your case @NatoNathan

NatoNathan commented 1 year ago

Would localhost works in your case @NatoNathan

Not really, I need to be able to target the docker host when running supabase locally and need to target supabase functions when we deploy it.

bombillazo commented 1 year ago

Yup, supabase hardcodes the webhook URL into the trigger definition, so any migration breaks the environment this was not build for...

marcfrankel commented 1 year ago

Jumping in to say this is also something we could use. It's hard to use db webhooks right now because the URL can't be dynamic based on environment variables

rafaelmagalhaes commented 12 months ago

has anyone come up with a temp solutions for this, facing some issues between development and production environments

jmwind commented 11 months ago

@sweatybridge I'm open to contributing a PR for to allow webhook config in the config.toml file. It's such a pain for local dev now that we depend on a few webhooks.

I was looking thru the CLI source and wonder if you could give me a pointer. I was going to add a new section to the template as defined in https://github.com/supabase/cli/blob/main/internal/utils/config.go but then wasn't sure what you suggest in terms of where to read the config and create the webhook db records into the webhooks table. Is there an example where config from config.toml turns into a few upserts into the db?

I'm assuming that webhooks are stored in a table, but can't find it in the local schema. There's the hooks audit table in the supabase_functions table but can't find the associated definitions.

bombillazo commented 11 months ago

@jmwind webhooks are stored hardcoded into the triggers, not in tables.

sweatybridge commented 11 months ago

url = "env(API_URL)" params = { Authorization = "env(JWT)" }

~I think it's tricky to add these configurable params via config.toml because we don't have any mechanism to push configs from local to remote project. What happens in case of conflict? Should it error, create new, or override existing webhooks?~

~Using data migrations would allow us to handle these cases using on conflict SQL clause so we don't need to reinvent the wheel.~ nvm I was confused.

webhooks are stored hardcoded into the triggers, not in tables.

This is correct.

The blocker here is to have variable api_url and anon_key separate between local, staging, and prod.

We have a RFC internally to expose these as database configs, for eg. select current_setting('supabase.project.url') should return the corresponding api_url for local, staging, etc. I will try to get it prioritised soon.

NatoNathan commented 11 months ago

The blocker here is to have variable api_url and anon_key separate between local, staging, and prod.

We have a RFC internally to expose these as database configs, for eg. select current_setting('supabase.project.url') should return the corresponding api_url for local, staging, etc. I will try to get it prioritised soon.

That definitely sounds like a really nice approach.

Another approach that could potential be implemented before that, is have the CLI generate and run the same queries that the studio runs to make and update webhooks. Although this would only fix this scenario. I have been meaning to investigate this solution and maybe even open a PR for a while but haven’t had the time.

Akash187 commented 10 months ago

An alternative is to create web hooks from studio UI and save them as as a new migration.

supabase db diff --schema supabase_functions -f my_webhook

Would the above work for you?

EDIT: nvm, we actually need to save the insert statements not the web hooks schema.

I don't think it will work for when you have staging and production environments.

jdgamble555 commented 10 months ago

The staging / dev issue could be solved with this:

https://github.com/supabase/cli/issues/1518

And just run Supabase based on your desired env file.

J

anthonyGustave commented 9 months ago

I coded something hoping that it could help you. What is sad is that i don't use supabase webhook in it but you need to create a new schema config and secure the schema. After connect the trigger to this code:

CREATE OR REPLACE FUNCTION public.new_sync_webhook()
 RETURNS trigger
 LANGUAGE plpgsql
 SECURITY DEFINER
AS $function$ 
DECLARE
    target_url TEXT;
    service_role_key TEXT;
    headers JSONB;
    payload JSONB;
    response TEXT;
    operation_type TEXT;
    record_type TEXT;
BEGIN
    -- Fetch URL and service role key from the configuration table
    SELECT config_value INTO target_url FROM config.app_config WHERE config_key = 'api_url';
    SELECT config_value INTO service_role_key FROM config.app_config WHERE config_key = 'service_role_key';

    IF TG_OP = 'INSERT' THEN
        operation_type := 'INSERT';
    ELSIF TG_OP = 'UPDATE' THEN
        operation_type := 'UPDATE';
    ELSIF TG_OP = 'DELETE' THEN
        operation_type := 'DELETE';
    END IF;

    record_type := TG_TABLE_NAME;

    IF OLD IS NOT NULL THEN
        payload := jsonb_build_object(
            'table', record_type,
            'type', operation_type,
            'record', row_to_json(NEW),
            'old', row_to_json(OLD)
        );
    ELSE
        payload := jsonb_build_object(
            'table', record_type,
            'type', operation_type,
            'record', row_to_json(NEW)
        );
    END IF;

    -- Concatenate the additional path to the URL
    target_url := target_url || '/{{your endpint here}}';

    SELECT extensions.http((
        'POST',
        target_url,
        ARRAY[extensions.http_header('Authorization','"Bearer ' || service_role_key || '"')],
        'application/json',
        payload::text
    )::extensions.http_request) INTO response;

    RETURN NEW;
EXCEPTION
    WHEN others THEN
        RAISE LOG 'HTTP request failed: %', SQLERRM;
END;

$function$
;

if you have any error you could see in the log of the container in local and debug it. For targeting supabase function you need to give the kong url

ykoitzsch commented 8 months ago

any update on this?

northfacejmb commented 8 months ago

Also looking for updates on this.

quememo commented 7 months ago

This would be really useful

lauri865 commented 7 months ago

Would love to be able to use environment variables in migration files (similar to config.toml) and define webhooks as migrations. Also, #501 could alleviate some pains by hooking up custom migrations workflows to the CLI.

coryetzkorn commented 6 months ago

Would love to be able to use environment variables in webhook HTTP request URLs. My requests currently get sent to a prod URL locally, leaving me with no way to test.

tecoad commented 4 months ago

@sweatybridge any update on this? I am struggling by working on a project locally with a team. Loading webhook endpoints from enviroment variables is very much needed. How are people currently being able to work with db functions on a team?

jdgamble555 commented 4 months ago

The only thing I can think of is to run a custom script, for example seed-vault.ts, which adds .env variables to the vault locally. Then use them in your functions.

Related to this:

https://github.com/orgs/supabase/discussions/12813

But having this built into the migration file would be best.

J

tecoad commented 4 months ago

@jdgamble555 from what i undestand, I am able to use these secrets stored in the vault from the trigger or functions. Right? But not able to find any documentation with that. I guess i have to create function which returns the secret, then call this function from the trigger right?

Somethin like this

CREATE TRIGGER "Profile update"
  AFTER UPDATE ON "public"."profiles"
  FOR EACH ROW
  EXECUTE FUNCTION "supabase_functions"."http_request"(get_webhook_url(), '{"Content-Type":"application/json"}', '{"PARAMETER_NAME":"PARAMETER_VALUE"}', '1000');

But not able to make this work. Am i heading to the right path?

jdgamble555 commented 4 months ago

I've never tested any of this so I couldn't tell you at this point, just theoretical.

ric-cardo commented 4 months ago

i was able to get it working using https://valentinprugnaud.dev/articles/setup-webhook-signature-with-supabase

aoztanir commented 3 months ago

Any update here?

AntonGrafton commented 2 months ago

Any updates?

ollebergkvist commented 2 months ago

Need this...

alexdlugo commented 1 month ago

This would be very helpful

clemwo commented 1 month ago

i was able to get it working using https://valentinprugnaud.dev/articles/setup-webhook-signature-with-supabase

link 404s unfortunately.

rymawby commented 3 weeks ago

i was able to get it working using https://valentinprugnaud.dev/articles/setup-webhook-signature-with-supabase

link 404s unfortunately.

I think this is the correct link mentioned above: https://blog.valentinprugnaud.dev/setup-webhook-signature-with-supabase