feature request: webhooks support

[capcom6]

Overview

Pulling changes by timer is not always the most efficient approach. Some services may update very infrequently, perhaps once a week or even less frequently.

To address this, it would be beneficial to add support for webhooks. Triggering the webhook endpoint would initiate an update of the stack.

Implementation Details

Webhook configuration should be implemented at the repository level. This feature could also serve as a foundation for manually triggering stack updates from the Web UI.

Related Issues

• Web UI

[EhabY]

The timer/polling approach is indeed inefficient but it was a way to get the UI up and running quickly. Triggering an update when an event from the server is received makes more sense. Though we probably have to batch these events in the server, in case they happen too quickly.

There are two ways straightforward ways to do this:

1. Websockets (duplex communication) - Which would make using other endpoints redundant since all communication can happen in the socket.
2. Server-Send Events (SSE) - Unidirectional communication.

SSEs make more sense to me since it's just a notification sent from the server to the client, and we can reuse already existing endpoints that other clients use (like a CLI) to request the new content.

[capcom6]

Sorry, I wasn't clear. I mean using webhooks to react to changes in a repository containing a docker-compose.yml file.

[sanzoghenzo]

This would be a nice feature to have!

Which events do you think we should handle? Is the push event enough?

Some remarks:

• each git service has its own webhooks with different payloads and headers, see github and gitlab, they may be similar but we have to take into account that the info we need can be in different places
• adding support for this, and for an API in general, would mean a complete rewrite of the core, since it is a simple loop over the configured repositories to check for changes.

@capcom6 following the discussion in #46: I don't want to discourage you in any way, and this is still my personal opinion, but it seems to me that this project is not what you need after all 😅 Let's see if @m-adawi proves me wrong!

[capcom6]

Any POST request to a specified endpoint should initiate a stack update. The payload can be arbitrary and is ignored.

The key aspect is that the webhook endpoint should be unique and unpredictable for each stack/repository.

Example: We have two stacks, A and B. After activating webhooks for each, we get unique URLs like:

• Stack A: /webhook/6bd565d4-1580-428b-bf68-0ac507c900cf
• Stack B: /webhook/2b8f5f0e-8b7c-4d3d-8c5b-1c0f8d0b0c0d

If any POST request is received at /webhook/6bd565d4-1580-428b-bf68-0ac507c900cf, stack A should be reloaded.

Since we don't analyze the payload, the webhook can be used with any external system that supports sending POST requests. It's not tied to Git. For instance, I could have a stack with the latest tag for an image and use a Docker Hub webhook to update the stack. This is useful because after pushing to the repo, it takes some time to build the image. A direct request from GitHub to SwarmCD might pull the previous version of the image.

While this project doesn't perfectly align with my requirements, it's the closest solution I've found. I prefer not to recreate what already exists. I considered forking the project, but I'm not comfortable with the current license.

[sanzoghenzo]

Thanks for the explanation; I never used webhooks so I thought it would need a single API endpoint + headers and payload parsing to determine the stacks to update 😅 it is definitely easier to implement it like that

[m-adawi]

I think a webhook is useful indeed. It can be integrated with GitHub Apps to make github send a request to SwarmCD informing it of new stack updates instead of having SwarmCD periodically pull for changes. But this also has security concerns, do you really want to publicly expose a tool that has full control of your docker socket?

Private webhooks can be useful too, for example if I want to press a button in the UI to make SwarmCD update a stack now instead of waiting the pull interval

[oliverlambson]

+1 for webhooks, think they would be a huge boost (the whole reason I came across swarm-cd is because I’m looking for “ArgoCD for swarm”)

the codebase looks quite understandable so if @m-adawi is open to PRs and has an opinion on implementation I’m happy to have a crack

[m-adawi]

@oliverlambson Yes PRs are welcome! Feel free to submit any contributions. Looking forward to it!