Protect the Kafka Rest Proxy API by using an HTTP proxy

[blootsvoets]

For example Nginx or Apache could be put in front of the Kafka Rest Proxy to restrict consumer paths, and the dashboard and API should preferably have HTTP basic authentication.

[fnobilia]

RADAR-CNS/RADAR-Backend#5 could be solved also with a HTTP Proxy

[afolarin]

Is there generally a preference for any of these three? Nginx HAProxy Apache

[fnobilia]

We need to develop at least a custom filter to ensure that each message contains both key and value. I would go for the easiest one in terms of development.

[blootsvoets]

HAProxy would not be suitable for processing and filtering. For custom filters, both Apache and Nginx by default support C code. Nginx has modules to also allow Lua, Java, Groovy, Clojure, JavaScript, Perl, with a variable performance hit (but very possibly much faster to develop). For basic path filtering like header checking, I'd vastly prefer Nginx because of its simplicity and readability. For the filter API, they look pretty similar. All-in-all, I'd choose Nginx.

For a content-checking filter for the Kafka REST Proxy, we could also modify their docker container with a different WEB-INF/web.xml combined with our own Java code. That is probably faster to develop than a webserver-based one.

[afolarin]

Copying Serge Osokin into this as I think it is a good starting point for discussion cc/ @sosokin

[nivemaham]

[WIP] We have a simple HTTP-Proxy-Servlet running on a tomcat in front of kafka-rest-proxy. I will add a new docker-compose with new tomcat integrated.

[blootsvoets]

Also nginx is now deployed. I'll set up the relevant location blocks.

[yatharthranjan]

I think its implemented now. Should we close this one ?

[afolarin]

yup