Scruid should support authentication

[barend]

The current (2.2.0) version of Scruid does not support authentication. Scruid should be able to authenticate itself with the Druid Broker.

Some thoughts:

• to my mind this is only required in the DruidAdvancedHttpClient that @anskarl recently added. The simple HTTP client can remain simple.
• supporting naive HTTP BASIC auth is the simplest case: it is sufficient to have an interception point in the outgoing half of the connection flow where an Authorization: header with static credentials can be blindly and synchronously added to the request.
• however, Druid's auth support is completely pluggable, so for Scruid to support every possible case requires a more complex interception API:
  • the interceptor should be able to trigger after the original "401 Unauthorized" response arrives from Druid, asynchronously contact a different endpoint such as a KDC or a Vault, obtain whatever credential it requires, and then re-send the original request with the necessary headers added.
  • to do so, it should probably exist as a long-living object (for keeping shared state, such as a token and its expiry time) that creates a per-request object associated with an invocation of the flow.

[barend]

I have no prior experience with Akka. Having looked at the code, I am confident that I can contribute a PR that implements the simple case (which would solve my immediate requirement). However, it would result in an API that cannot support the complex case.

If I can get a little help in defining the interception API and integrating it into the DruidAdvancedHttpClient at the appropriate places, I'd be happy to do the implementation work to get the fancy version in place.

[bjgbeelen]

Hi @barend ,

I dug into the AdvancedHttpClient a bit more and together with the cases you described, I implemented a rough design which I think would work for your case. It's on the outer edges of the executeRequest, not touching any of the internal stream stuff.

I have a branch here: https://github.com/ing-bank/scruid/compare/request-flow-customization

As you know I don't have much time coming weeks, so I hope it is of help in further steps to take. You can continue on the branch, or just use it as an inspiration, or go in a completely different direction.

[anskarl]

Hi @barend and @bjgbeelen

I would like to help regarding the authentication support in Scruid. @bjgbeelen I took the initiative to make some additions to your design in order to add support for configuration and prepare a testing environment.

• At the moment I've added a docker image with nginx as a basic-auth proxy in front of Druid. I am using the image beevelop/nginx-basic-auth:latest. The image forwards requests to Druid, but it expects to perform basic-auth first. The username and password are given to the env variable HTPASSWD (the value is created using htpasswd -nb user aloha).

• In tests, the ing.wbaa.druid.AuthHttpClientSpec overrides the configuration and sets BasicAuthenticationAddition to the DruidAdvancedHttpClient in order to perform a typical time-series query with basic-auth.

• I haven't made any test for Kerberos at the moment and I haven't add a Kerberos service to the docker-compose. Perhaps an option is to use an image like gcavalcante8808/krb5-server:latest but we should also change the docker-image of Druid, in order to set-up parameters for Kerberos (https://druid.apache.org/docs/latest/development/extensions-core/druid-kerberos.html)

All additions are in https://github.com/anskarl/scruid/tree/request-flow-customization

[bjgbeelen]

Awesome @anskarl ! I applied some renaming this morning, saw you worked on the branch already before i made those :D (i think I like RequestFlowExtension better than RequestFlowCustomization

I think @barend will find these additions very welcome!

I'll try to keep an eye on the development that's going to happen, but I'm expecting a daughter any time now, so will have focus off of my computer

[anskarl]

Excellent news @bjgbeelen! I wish all the best for the birth of your daughter!!! I am also expecting a daughter the next month :)

I will update the branch with the changes and rename ing.wbaa.druid.AuthHttpClientSpec to ing.wbaa.druid.AuthenticationSpec in tests.

[barend]

I have added another fork in the mix: barend/request-flow-customization 😄

No real customisations of my own yet. This starts with anskarl/request-flow-customization, merges the latest changes of master, and moves the authentication proxy into the existing NGINX container. I did add a "negative" test for the authentication flow, so that's a start. I'll look into this in more detail.