Refactor Shibboleth Authentication workflow

[simong]

We should look into refactoring our current shibboleth authentication workflow. It's now possible to run the shibboleth SP software in combination with nginx (over FastCGI) [1]. Moving all that logic to the web nodes would allow us to:

• Move to a single Service Provider deployment, rather than a SP-per-tenant
• Register that SP with federations such as InCommon, eduGAIN, .. (this would be a HUGE win as institutions their IdP would no longer need to add us, they would grab it from the federation)
• Able to re-use the Shibboleth SP software and get rid of the custom SAML module
• We would be able to support Where-Are-You-From-Discovery (WAYF)
• No Java requirement for the app servers

A downside is that we'd no longer be able to add shibboleth authentication "on the fly" in the admin UI. This doesn't seem like a big issue to me as in the past it wasn't functioning untill the institution's IdP added our metadata.

[1] https://wiki.shibboleth.net/confluence/display/SHIB2/Integrating+Nginx+and+a+Shibboleth+SP+with+FastCGI

[nicolaasmatthijs]

@simong : Why would we lose the ability to configure shibboleth authentication on the fly? How it practically work for an institution in one of the federations to set up their tenant?

[simong]

@nicolaasmatthijs

You would still be able to enable shibboleth authentication but not configure it on the fly, as most of the configuration has moved to the Shibboleth SP software and Apache.

If an institution wants to use OAE with their Shibboleth IdP, 2 situations can exist (assuming we're part of the UK federation/eduGAIN):

1. The institution is part of the UK federation/eduGAIN. Unless we want more attributes, nothing needs to happen. Their IdP will automatically trust us due to both of us being part of the federation.
2. The institution is NOT part of the UK federation/eduGAIN. Again, there are 2 things we can do. 2.1 If the institution is part of a federation we're not part of, we could try to register our SP with that federation 2.2 If the institution is not part of any federation, they will have to "trust" our SP. Which means their IdP needs to acquire our metadata.

If an institution wants to use Shibboleth, we need to know 2 things:

• What the entityID of their Identity Provider is (we can usually find this in the uk federation metadata)
• What attribute we should use to identify a user. (eppn, cn, ..)

Once we know these 2 values, we will have to:

• Add that information in the puppet scripts
• Run puppet on the web0 node. Puppet will:
  • create an application override in the shibboleth config for that tenant which specifies what attribute will be used to identify a user
  • create a vhost in apache for the tenant's domain name, so we can tell shibboleth which tenant we are (so shib knows which application override to user)
• Enable shibboleth authentication in OAE and configure the entity ID of the identity provider (so OAE can tell shib where it should send users upon authentication)

This means we'll lose a little bit of flexibility, but considering:

• we are using the proper shibboleth software over established protocols
• no longer parsing signed SAML requests
• we are able to easier deal with federations
• we are able to support WAYF/Discovery
• we are able to support older shibboleth/saml protocols
• we now have a single SP rather than a SP-per-tenant
• we used to have to wait on IdPs anyway (to approve the SP)

I feel that it's worth it.

I have a PR pending at https://github.com/oaeproject/Hilary/pull/849. It's probably best if we discuss it there further.