Credentials Class Refactoring

[BrunoCoimbra]

We're refactoring the GlideinWMS Credentials class to improve our code and introduce new features. This issue is used to keep track of our progress and receive requests and suggestions from the project's stakeholders.

New features and improvements

• [ ] Give more control to operators on how Frontend IDTOKENs are generated
• [ ] Allow operators to specify which credentials are sent where
• [x] Add the possibility to send arbitrary credentials to specific sites
• [x] Treat credential types and functions (what they’re used for) separately
• [ ] Factory entries will support a list of required and optional credentials instead of a single auth_method
• [ ] Improve the way the "fair split" method handles multiple credential types

Bug fixes

• [ ] Fix an issue that prevents credentials already sent from being updated after a reconfig
• [ ] Fix an issue that causes the HEPCloud Decision Engine to stop updating classads when no jobs are queued
• [ ] https://github.com/glideinWMS/glideinwms/issues/290

Work Notes

### Tasks
- [ ] https://github.com/glideinWMS/glideinwms/issues/291
- [ ] https://github.com/glideinWMS/glideinwms/issues/294
- [ ] https://github.com/glideinWMS/glideinwms/issues/295

[jasoncpatton]

• [ ] Give more control to operators on how Frontend IDTOKENs are generated

Using an old, patched version of the Frontend to get IDTOKENs working with my collaborators' Collectors, I happened to use a sub claim of gwms-{glidein_site}@$(TRUST_DOMAIN) (and so had them set ALLOW_ADVERTISE_STARTD = gwms-*@$(TRUST_DOMAIN) on their collectors). Now that I'm finally upgrading these Frontends, I'm applying a small patch here https://github.com/glideinWMS/glideinwms/blob/8f1d220eab44b2c1eeffd33882736c7ff7f828fc/frontend/glideinFrontendElement.py#L1138 to set identity = f"gwms-{glidein_site}@${TRUST_DOMAIN}" (where ${TRUST_DOMAIN} is filled in per frontend). As mentioned in the TODO item, it would be nice if there was some more control here in the config.

I suspect allowing the config to template with {glidein_site} might be enough to start here, something like:

        self.idtoken_subject = self.elementDescript.merged_data.get("IDTokenSubject", "{glidein_site}@{socket.gethostname()}")
...
                    identity = self.idtoken_subject.format(glidein_site=glidein_site)

Then the config could be:

   <security
      security_name="myvo"
      idtoken_subject="gwms-{glidein_site}@my-vo-collector.domain"
   >

I am also already able to set the TRUST_DOMAIN in the condor config used by the Frontend to set the iss claim, though being able to configure that in the config would also be useful.

[jasoncpatton]

Hi again @BrunoCoimbra! To summarize what we talked about after your fantastic HTC23 presentation:

1. I think it would be great if credential purposes could be defined arbitrarily (outside a set of reserved purposes, e.g. ce_auth for glidein->CE, cm_auth for glidein startd->pool collector, etc.). For example, I could imagine a VO requesting a bunch of different tokens/proxies/etc. being used for testing a bunch of different storage services.
2. For credential discovery, any well-documented method is fine but perhaps the simplest method would be to name the credentials based on their purpose and (as I'm now realizing might be needed) credential type, e.g. ce_auth.scitoken, ce_auth.proxy, cm_auth.idtoken, cvmfs_auth.scitoken, cvmfs_auth.proxy. (Maybe credential type would also be arbitrary with some reserved types.)