Add Webhook-Ingest service using NATS JetStream

FelixTJDietrich commented 1 month ago

Motivation

We want to have a reliable way of receiving webhook messages from GitHub etc. Additionally, we want to subscribe to past and future webhooks also for development porposes.

Description

Add a webhook-ingest FastAPI service that can receive webhooks from GitHub and publishes them to NATS JetStorm.

We will host this soon on a server to collect GitHub data already. JetSteam will persist the messages and send them to any subscriber in the future.

Additionally, this PR adds a VSCode workspace since this was the only way of getting the venv to work for me with IntelliSense etc.

Testing

I hosted the NATS server temporarily on tls://nats.heph.felixdietrich.com with access token asdf, it receives all webhook events from the Hephaestus repo.

Install natscli (Repo)

brew tap nats-io/nats-tools
brew install nats-io/nats-tools/nats

Examples:

View past webhook data: nats --server tls://asdf@nats.heph.felixdietrich.com stream view github

Subscribe to incoming webhooks and then interact with the Hephaestus Repo: nats --server tls://asdf@nats.heph.felixdietrich.com sub "github.>"

Screenshots

Using nats --server tls://asdf@nats.heph.felixdietrich.com stream subjects:

Checklist

General

[x] PR title is clear and descriptive
[x] PR description explains the purpose and changes
[x] Code follows project coding standards
[x] Self-review of the code has been done
[x] Changes have been tested locally
[x] Screenshots have been attached (if applicable)
[ ] Documentation has been updated (if applicable)

Server

[ ] Code is performant and follows best practices
[ ] No security vulnerabilities introduced
[x] Proper error handling has been implemented
[ ] Added tests for new functionality
[x] Changes have been tested in different environments (if applicable)

FelixTJDietrich commented 1 month ago

test comment :)

FelixTJDietrich commented 1 month ago

I think it is ready now, debugged forever why it was not working with authentication. The reason was Python 3.9 and that's why I switched to 3.12

FelixTJDietrich commented 1 month ago

Will just merge this and host it on our VM

ls1intum / Hephaestus