Detect and allow connection with an existing JWT token inside header

[lioneloh]

We should allow the okdp-spark-auth-filter to rely on an existing JWT token provided inside a header instead of asking the user to authenticate again. It will allow authenticating reverse-proxy or gateway to authenticate the users and proxy the HTTP request to the spark UI with identity propagation.

Consider this use case :

We need to have inside the JWT token all information needed (sub,email,groups claim, ...).

This Gateway could also avoid setting individual redirect URL for all SparkUI. Instead the Gateway will host the only redirect URL for all SparkUI

What do you think ?

I may help if we all agree on the scope.

[idirze]

Hello @lioneloh,

Thank you for the proposal, it will be a nice feature to have natively in the filter. So, please go ahead if you have time.

For information, we have planned to develop the gateway part (UI + Rest API) authenticated with OIDC. The gateway will discover automatically all the spark jobs running or completed in the cluster and list them in the UI (links, status, start/end time, etc). There will be one single URL for all spark jobs listed in the UI (Gateway URL). When the user clicks the url, the gateway renders the live spark UI if the job is still running or renders/redirects to spark history when the job was completed (failed, terminated). The gateway will also come with other features like monitoring, logging, optimisations, etc.

Your feature brings flexibility in the usage of the filter through a reverse proxy or an own existing gateway.