Python lambda for sending well-defined messages to confluent kafka assumes AWS Deployment with API Gateway exposure of endpoint
Hearth of the solution lies in the Src folder
POST π method is guarded by JWT token in standard header "bearer"
Method | Endpoint | Info |
---|---|---|
GET | /token |
forwards (HTTP303) caller to where to obtain JWT token for posting to topic |
GET | /topics |
lists available topics |
GET | /topics/{topicName} |
schema for given topic |
POST π | /topics/{topicName} |
posts payload (after authorization and schema validation) into kafka topic |
POST | terminate |
kills lambda - useful for when forcing config reload is desired |
There are 3 configs for this solution (in conf folder)
Whole solution expects to be deployed as lambda in AWS, there are prepared terraform scripts to make initial deplyoment, and can be found in "terraform" fodler All that is needed is supplementing variables for
Once tfvars are supplied, go terraform apply and you are done
Useful scripts for dev and Deployment
Jupyter notebook, with one cell for lambda initialization and one cell per method, for testing purposes Obviously using it requires correct configs to be in place (PUBLIC key is being loaded during initilization)
shell script for fetching pithon requirements and ziping it together with sources and config into lambda archive, ready to be used by terraform