SAML Module Upgrade Failure Investigation and Path Forward

[olivereri]

Description

July 20th's SAML module upgrade ended in failure. This issue will investigate the failure and develop a path forward to an eventual successful deployment. The previous issue is referenced in "Related issues".

Acceptance Criteria

• [x] Technical path forward determined
• [ ] Technical solution is implemented and D10 upgrade unblocked by phpSimpleSAML

Related Issues

https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14246

Implementation Details

The PIV logon failures for prod.cms.va.gov are captured in the below error log:

This error indicates that the public certificate installed to validate signed responses from the Identity Provider are invalid. On the night of the upgrade, in coordination with the Identity and Access Management (IAM) team, the public certificate was confirmed as correct. The suspicion now is that the response was being signed by a different key that doesn't match the public certificate installed.

On July 24th a test on CMS-Test Staging was conducted by installing a valid but incorrect public certificate for signature validation:

The error response, while having a different severity, is the same as observed on July 20th.

Team

Please check the team(s) that will do this work.

• [x] CMS Team
• [ ] Public Websites
• [ ] Facilities
• [ ] User support
• [ ] Accelerated Publishing

[olivereri]

I've reached out to the IAM team. They understandably have reservations about integrating test.staging.cms.va.gov. They had suggested creating a new integration and we can create a new server/environment. I mentioned that we have test.prod.cms.va.gov so we may go that way.

I did ask whether or not the response was signed by a different private key and they said now. Stating that: There are a total of 103 federation partners in Partnership model (which we are using for CMS) that are currently active in PROD (including current CMS integration) without any issues. I get the sense they think this problem sits chiefly on our side.

[edmund-dunn]

Talked to the maintainers. They concur it was most likely the cert.

[mchelen-gov]

To clarify, the upgrade here refers to moving from https://www.drupal.org/project/simplesamlphp_auth to https://www.drupal.org/project/samlauth right?

[olivereri]

To clarify, the upgrade here refers to moving from https://www.drupal.org/project/simplesamlphp_auth to https://www.drupal.org/project/samlauth right?

@mchelen-gov That is correct.

[olivereri]

Through a series of e-mails the Identity and Access Management team has agreed to integrate test.prod.cms.va.gov with their Prodcution IdP. They are waiting for us to respond that we are ready and provide the metadata for the application.

[olivereri]

@olivereri @mchelen-gov @EWashb @teeshe @edmund-dunn @BerniXiongA6 met to discuss SAML troubleshooting efforts and possible paths forward. We came to the conclusion that SAMLAuth testing should be done to reproduce the issue or be used to learn from a working standard. This is instead of progressing with miniorange's module using SimpleSAMLphp because there are blockers due to funding lead times and a trail period shorter than we need.

Notes from the discussion: https://vfs.atlassian.net/wiki/spaces/PCMS/pages/2722988150/Blocker+Drupal+10+Upgrade+with+SAML+and+SSOi

[productmike]

Shifting back to READY FOR SPRINT since this is not committed work for S91. Can be pulled in if timelines are shorter than expected.

[EWashb]

Can we get an update on these tasks? I know we went with one SAML module over another so I'm unsure if all of these tasks are still appropriate for our path forward @teeshe @edmund-dunn cc: @BerniXiongA6

[edmund-dunn]

Closing this ticket. Simplesamlphp update was tested successfully on test.prod.cms.va.gov.