We need a basic generic webhook AS

[matrixbot]

Created by @ matthew:matrix.org.

[ara4n]

I tried to bring this up on #matrix-core:matrix.org and largely failed.

My rationale for a generic webhook AS is to provide a trivial API for injecting & receiving messages into/from a matrix room via HTTP in a way that resembles the webhook pattern used by Slack / MatterMost / RocketChat / Github etc - thus making it easier for folks to port existing Slack-etc bots to Matrix. However, rather than cloning the Slack API verbatim, one would be able to send/receive actual Matrix events rather than dumbing them down into Slack events.

The vague idea was:

• Let API users register a URL (hook) that gets called by the AS when an event happens in a room.
• Provide a webhook idiomatic endpoint that API users can hit when they want to send a message into the room. Slack/MM/RC/GH all seem to handle idempotency (if at all) with txnids in the request body rather than the URI, so feels like it could be good to follow the same pattern. Alternatively, one could just use the normal CS API.
• We could possibly allow messages inbound to the room masquerade user_id (similar to slack's "bot" messages), although this could be a security concern; might want to use personae/profiles or something else more equivalent to Slack's "BOT" labels.

Anything more sophisticated than sending/receiving messages would be a case for the real CS or AS API.

@aviraldg is already working on a Slack webhook API AS at https://github.com/aviraldg/matrix-appservice-slack-api; there is significant overlap between that and this idea, so any concerns about this should be raised so that they can feed into his work.

Finally, there seems to be a question on whether this is best positioned as a separate AS or instead as an evolution of the core AS API. Justifications one way or the other welcome.

[Half-Shot]

A few ramblings below

Provide a webhook idiomatic endpoint that API users can hit when they want to send a message into the room. Slack/MM/RC/GH all seem to handle idempotency (if at all) with txnids in the request body rather than the URI, so this seems useful. Alternatively, one could just use the normal CS API.

Something bothers me about this. I'm presuming there will be some authentication when this is called. I feel like the CS API would be more suited to this. Is there a use case when you wouldn't be able to use the existing API to send messages?

Finally, there seems to be a question on whether this is best positioned as a separate AS or instead as an evolution of the core AS API. Justifications one way or the other welcome.

My view on this entirely depends on how powerful you want it to be. If we're just pushing events out to services then I think an AS service is more appropriate rather than trying to complicate the CS/AS API any further.

I'm not sure it's a good fit for the AS API, so if you were going to spec a API for webhooks, I'd rather it was a new one entirely. From my understanding, the AS API is designed for tight integration between the HS and a particular service which is trusted to create users, rooms and do anything except modify events while a webhook would presumably simply be reading and writing events from a predefined room (and if you wanted to do more than that, you're getting dangerously close to just cloning the CS API once you star).

[kegsay]

tl;dr - I like the idea of a webhook API, but I feel this should be done by the HS rather than being bolted on via a webhook AS.

I feel we need a better reason to do this over and beyond "because that's what Slack/MM do". We already have a push-based method to get events out of an HS, it's called the AS API. We already have a trivial HTTP API to send messages, it's called the CS API. I do not want to have another API. Let me stress that I'm not against a webhook API, but I strongly feel it should be focussed on giving devs something that the other APIs are lacking (e.g. the ability to remotely register webhooks).

I have concerns that the desired feature set outlined in the bullet points above are simply not possible to do in an AS:

Let API users register a URL (hook) that gets called by the AS when an event happens in a room.

• How does the AS know you are who you say you are? What's stopping me from saying I am @matthew:matrix.org?
• Assuming that is resolved, how does the AS sniff every single event for that user account (so it can invoke the webhook when an event happens in the room)? You can't remotely register new regexes, so you'd have to register for .* - which makes the AS a huge honeypot (this was one of the core reasons why we didn't want to allow remote reg: because a compromised AS might listen for .*!).
• Because it is now receiving every event, it needs to know when to invoke the webhook. It can't just invoke it for every incoming event for that room ID because that user may not be in the room. Now you start modelling room permissions to know if you should be passing events through to the webhooks.
• etc..

A lot of these problems come from having to restrict events for a given user. Let's assume a trusted user:

Let API users register a URL (hook) that gets called by the AS when an event happens in a room.

• How is this any different from AS API registration (other than it being remote)? No alias/user/room regex? That's the same as registering for .* and then filtering based on some other criteria (room ID, event type, etc) which should arguably then be a feature of the AS API.

[leonerd]

(copypaste of what I said on Matrix Core): For URLs to that bots/scripts/etc.. can use to send messages in to Matrix, they should be using the normal CS API for posting messages. If that API is incapable currently then we should look into fixing that until it is

On the subject of messages out, I think being able to tell my server "please poke out on this URL when these kinds of messages happen". That's a generic push server. I don't think the core HS program (synapse) itself necessarily ought to be doing that, for the same reasons we split the push service out.

[Valodim]

I'd like to bring this up again. It seems to me like there is little to do here server-side:

• the CS api offers simple enough submission of events
• the push notification api offers a way to receive http callbacks from the server, with a ton of filtering flexibility even

The desired user experience requires two things:

1. A simple, user-facing way to obtain a URI to send messages to
2. A simple, user-facing way to register locations to send push events to

The first point is dealt with by showing the user a URI that includes a valid access token. So a logged-in client needs a way to generate a new valid access token. Bonus points if a list of such tokens can be displayed, associated with a description, revoked individually, or maybe even given a simple set of permissions.

The second can be handled with an interface that allows listing and creation of pushers. Bonus points for an associated custom description, and request body template.

All in all, this seems like a pretty good deal in terms of usefulness vs complexity. Am I missing some bigger issue here? Maybe exposing the APIs in this way has too much potential for abuse, or results in too much strain on servers?

[ptman]

I wrote a simple go service that can accept POST requests like the ones described in https://api.slack.com/incoming-webhooks and send matrix messages to channels based on those. It could help until matrix gets better support in various tools

[aviraldg]

Paul, I already wrote that for go-neb. It's not very well documented though.

On Fri 21 Jul, 2017, 12:18 AM Paul Tötterman, notifications@github.com wrote:

I wrote a simple go service that can accept POST requests like the ones described in https://api.slack.com/incoming-webhooks and send matrix messages to channels based on those. It could help until matrix gets better support in various tools

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/matrix-org/matrix-appservice-bridge/issues/6#issuecomment-316795695, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOlRA-091f_9KgC8odHgrFkieeyP7e_ks5sP6ENgaJpZM4IpWYE .

--

Regards, Aviral Dasgupta

[cavebeat]

Hi,

I am searching for a way to tell my homeserver to send events (messages) from a specific room to an URI.

the push notification api offers a way to receive http callbacks from the server, with a ton of filtering flexibility even

This sounds like the thing.

has there been any progress, is this available?

how can i achieve this now? registering an appservice for a bridge/bot to send events to a webhook?

br cave

[ptman]

@cavebeat No need for appservice. Just login and join the room and get room events: https://matrix.org/docs/spec/client_server/r0.2.0.html#get-matrix-client-r0-rooms-roomid-messages

[Evidlo]

Perhaps this should be closed now:

turt2live webhook bridge

[turt2live]

Author of that bridge here. What's described above is something a lot more involved and baked into the API. Comparing my bridge to the discussion here:

• The bridge skirts the issue of masquerading by generating a (horrid) user ID in the _webhook namespace.
• The bridge doesn't accept plain matrix events, and instead relies on a limited set of custom fields
• The bridge does implement most of the Slack API the best it can given the constraints of matrix (see issues on the repo labelled "slack compat" and "lacking matrix spec")
• The bridge is an appservice - there's a few comments above about how the webhook API should be baked into matrix instead of being an appservice.

I'm a bit biased, however I like the idea of the webhooks side of things being an appservice. I find it easier to bolt on functionality than to support it all the time. If this kind of API were to end up in the spec then homeserver authors would end up having to support another api. Instead, they could redirect their efforts to making an incredible appservice api.

[Half-Shot]

To be honest, I believe this kind of functionality is best served by another library or bridge and not attached to this project. I'm going to close this, unless anyone has any itchings to work on it.