search

Sustainsys / Saml2

Saml2 Authentication services for ASP.NET
Other
951 stars 601 forks source link

Signature not verifying on certain AzureAD accounts #1349

Closed Shumiii closed 2 years ago

Shumiii commented 2 years ago

I am using Azure AD as the IDP and I am getting the error: Signature didn't verify. Have the contents been tampered with?

This only happens on certain accounts - on most accounts it works fine.

This is from the SAMLResponse from Azure that gives the error:

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
    <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
    <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
    <Reference URI="#_09095fa6-55d9-4672-ade7-c1c696298200">
        <Transforms>
            <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
            <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
        </Transforms>
        <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
        <DigestValue>yOap8j7tUqwqsEuqMgXcbzD0ry4yKfKnOf1TuWDbrXw=</DigestValue>
    </Reference>
  </SignedInfo>         
<SignatureValue>G9jCJJN6aDmJqhciN1nF3lvkq2+GCaPYegTsxiVTtyqwrFTyQYKJggyVdDAiuG3mqxhZsE2Lg+8CXaafOLRfU5B3IdU8n625JCSAkXfEu8OcFu9NuOlUVFSlJc5iGBEP9uIRmvimZqljz0CmoJgun1hB38NB8aWEZTmUVRdiMUz4dgYUzTpkuMhhfGBEPNu7YfvSXgzYdLAfFrt6yJS+Hjbp7fiP34ulS7qT2VFTN1Ga/rPN/WrnUXuq3y2eceh3X4vTw/owkbtHKYwLOdhH02pYHL9oUg/Na6wSc+QJeu6ECQbICvnB7CwnuU+5BcDDI5MZuC1ghXUvoj1gS+nQEw==</SignatureValue>

This is from a SAMLResponse that works:

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
  <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
  <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
  <Reference URI="#_050e9600-9a36-4b29-86a1-917b135a5f00">
      <Transforms>
          <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
          <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
      </Transforms>
      <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
      <DigestValue>VId+pp2DPWXrgQdKxZtIuA67oiLtyy18JKAYPU5o7rA=</DigestValue>
  </Reference>
  </SignedInfo> 
<SignatureValue>aNFs933L0y1Qko5d55/Ohmys5ucwAGPTPXQ9jhCdJ/CUXvERdfI7zGGDLm1sT0vXLJRKKU/8vWhdyNx8YPIwdEzqVeC5xRZVDA0BI3JbSEs3c8sZ9+ap/eJrMy5z7j/7Bhm2YcJUeF51/0jaHd9GSvWt9t3nDuDwO6bjfCQ63qUUK1jFLIOBeAGuQjz55aUjgIxU9xe+qOAuf+X1XPvwDMuf90H2kzuRg3cSCDbwpSrBABBEGg5V0CFhds5T3Ldi4fWMmzCjOT6rO85PR1D4yegVtg5G48r4KBC0fKJtkGxaXkWlJQZaC13SVonPmsjB7LbuIqm7AI/a9nK2ZA9tNA==</SignatureValue>

Would appreciate any help or pointers on how to further investigate the issue.

AndersAbel commented 2 years ago

There has been similar bugs reported previously and when they were investigated we found indications that some special characters where causing this. What does the data in the payload look like? Is it English using standard ascii characters or something else? Can you see any pattern on special character contents between working and non-working responses that could indicate the problem is caused by certain characters?

Shumiii commented 2 years ago

Thanks @AndersAbel. I have been analysing for the special characters but I do not see anything remotely unusual. Would you mind casting your eyes over this response for special characters? I have modified here and there for privacy reasons but I only swapped text with text. Is there a list of special characters I can search and check for? 


<samlp:Response ID="_48f73815-64d5-4fba-ab52-22daf6ed93ca" Version="2.0" IssueInstant="2022-05-16T13:23:10.738Z" Destination="https://site.com/Saml2/Acs" InResponseTo="id93b45142abb24ccf83777e91e6a025fa"
    xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol">
    <Issuer
        xmlns="urn:oasis:names:tc:SAML:2.0:assertion">https://sts.windows.net/3401317f-bda5-49d1-8668-9d3caae9970a/
    </Issuer>
    <samlp:Status>
        <samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" />
    </samlp:Status>
    <Assertion ID="_09095fa6-55d9-4672-ade7-c1c696298200" IssueInstant="2022-05-16T13:23:10.738Z" Version="2.0"
        xmlns="urn:oasis:names:tc:SAML:2.0:assertion">
        <Issuer>https://sts.windows.net/3401317f-bda5-49d1-8668-9d3caae9970a/</Issuer>
        <Signature
            xmlns="http://www.w3.org/2000/09/xmldsig#">
            <SignedInfo>
                <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
                <Reference URI="#_09095fa6-55d9-4672-ade7-c1c696298200">
                    <Transforms>
                        <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
                        <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                    </Transforms>
                    <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
                    <DigestValue>yOap8j7tUqwqsEuqMgXcbzD0ry4yKfKnOf1TuWDbrXw=</DigestValue>
                </Reference>
            </SignedInfo>
            <SignatureValue>G9jCJJN6aDmJqhciN1nF3lvkq2+GCaPYegTsxiVTtyqwrFTyQYKJggyVdDAiuG3mqxhZsE2Lg+8CXaafOLRfU5B3IdU8n625JCSAkXfEu8OcFu9NuOlUVFSlJc5iGBEP9uIRmvimZqljz0CmoJgun1hB38NB8aWEZTmUVRdiMUz4dgYUzTpkuMhhfGBEPNu7YfvSXgzYdLAfFrt6yJS+Hjbp7fiP34ulS7qT2VFTN1Ga/rPN/WrnUXuq3y2eceh3X4vTw/owkbtHKYwLOdhH02pYHL9oUg/Na6wSc+QJeu6ECQbICvnB7CwnuU+5BcDDI5MZuC1ghXUvoj1gS+nQEw==</SignatureValue>
            <KeyInfo>
                <X509Data>
                    <X509Certificate>MIID4DCCAdigAwIBAgIQWHO0DpPCs4dPUkVmEiXIAjANBgkqhkiG9w0BAQsFADA0MTIwMAYDVQQDEylNaWNyb3NvZnQgQXp1cmUgRmVkZXJhdGVkIFNTTyBDZXJ0aWZpY2F0ZTAeFw0yMjAxMjYxNDE2MTBaFw0yNTAxMjYxNDE2MTBaMDQxMjAwBgNVBAMTKU1pY3Jvc29mdCBBenVyZSBGZWRlcmF0ZWQgU1NPIENlcnRpZmljYXRlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqapRE6K9yAUY8IahIUM0EjipJWMilvEIm7KA4d5C/Y/d/SEzKC42N7MYPLl393PjFVfqImuDqC8D+DbglMgEkB4gUvcowChxpgpcIUx98bCJ2DjnQZwWSan4lDzUePe5YqxY9mPhcPtPmSiec1IdZ9WotXnLFyz1ZDSFRqzA30aY8IA5yhZpeoOpeTNYhKlLrMz1ZrSXXAO8vulSQhr9atuop2w2ncF/4lWCT4Gp4lSi/9QgEe6dqTxwxXGBnqe3Ojtuz6D8/Zs113Vox7nhIhNRZnQggOMB18eK9CilIC0Qi4194eqBHo4HCkdmR2ncxWFIUJdH+XCU6g8SaU62NQIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQAHHKbR7jwnr8Res8CG21Cu5rpxiG+e0ca5nzJix1ccZBQlG+yH+JTgP28GLOmoviobcNhwYOMxxx1L1nbGegILeSyj0nZ5B5Ke+lD/zS5yy/2yJ6qosYxcaJhsNmRhx1Xh5LEpEni4xBVkrKMPdjMLjOeif6hTRKXBsdh7eoJq9/mO2tv/yKbeHDqFv6dVqIptMzJX7UU8hoVgxueby9XDl94ZuxAodeqmG+T+0+xBCOpMvZ4E9U/CbaZA25PFbmHXexhvTAhXNBzV3t94JEf5vOLwmzfM2gufGsE4C/rLMySl2vgxjunMBRoyhNYEUP+C4zuCB07gHZRmEmMSdnig</X509Certificate>
                </X509Data>
            </KeyInfo>
        </Signature>
        <Subject>
            <NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress">first.last@site.com</NameID>
            <SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
                <SubjectConfirmationData InResponseTo="id93b64142abb24ccf83777e91e6a025fa" NotOnOrAfter="2022-05-16T14:23:10.613Z" Recipient="https://site.com/Saml2/Acs" />
            </SubjectConfirmation>
        </Subject>
        <Conditions NotBefore="2022-05-16T13:18:10.613Z" NotOnOrAfter="2022-05-16T14:23:10.613Z">
            <AudienceRestriction>
                <Audience>https://site.com/Saml2</Audience>
            </AudienceRestriction>
        </Conditions>
        <AttributeStatement>
            <Attribute Name="http://schemas.microsoft.com/identity/claims/tenantid">
                <AttributeValue>3401317f-bda5-49d1-8668-9d3caae9970a</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/identity/claims/objectidentifier">
                <AttributeValue>4198eebf-5e28-47b0-99c7-bea51084611c</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/identity/claims/identityprovider">
                <AttributeValue>https://sts.windows.net/3401317f-bda5-49d1-8668-9d3caae9970a/</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/claims/authnmethodsreferences">
                <AttributeValue>http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password</AttributeValue>
                <AttributeValue>http://schemas.microsoft.com/claims/multipleauthn</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress">
                <AttributeValue>test@test.com</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/companyname">
                <AttributeValue>test.com</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/department">
                <AttributeValue>IT - Support</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/jobtitle">
                <AttributeValue>2nd Line Team Lead</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/lastname">
                <AttributeValue>Last</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/firstname">
                <AttributeValue>First</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/streetaddress">
                <AttributeValue>DIRFT 5  Dans Way</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/state">
                <AttributeValue>Northamptonshire</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/country">
                <AttributeValue>GB</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/officelocation">
                <AttributeValue>Daventry (Dist)</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/city">
                <AttributeValue>Crick</AttributeValue>
            </Attribute>
        </AttributeStatement>
        <AuthnStatement AuthnInstant="2022-05-16T13:20:53.007Z" SessionIndex="_09095fa6-55d9-4672-ade7-c1c696298200">
            <AuthnContext>
                <AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:Password</AuthnContextClassRef>
            </AuthnContext>
        </AuthnStatement>
    </Assertion>
</samlp:Response>
AndersAbel commented 2 years ago

Ok, looking at your example I guess it's English-language users we are talking about. The examples I recall was with Chinese characters.

Is the behaviour consistent in that it never works on certain accounts while other works? Or does it work sometimes and sometimes not for the same account?

Shumiii commented 2 years ago

Hi @AndersAbel, yes its English-language users. The behaviour is consistent in that it never works on certain accounts. Thanks for your time.

AndersAbel commented 2 years ago

Any chance you can share an intact SAML Response that doesn't work via some private channel (e-mail if that is ok for you)? Although it is always best to avoid sharing PII, troubleshooting this will be hard without a sample response that fails.

Shumiii commented 2 years ago

@AndersAbel sure I can. How can I email you please?

AndersAbel commented 2 years ago

@Shumiii thanks. E-mail at anders@sustainsys.com

AndersAbel commented 2 years ago

Thanks @Shumiii for the sample. I've run it through some local tests (see test program below). It looks like the hashing of the contents ends up with a different value when I use the SignedXml code that does the validation than the value found in the XML file. To verify I've done my tests right, can you please try running the program with a few more samples? Especially validating with a SAML Response that works would be great, to validate that the test program is indeed doing the right calculation:

var xd = new XmlDocument()
{
    PreserveWhitespace = true
};

var fileName = "C:\\temp\\non-validating-response.xml";

xd.Load(fileName);

var fileContents = File.ReadAllText(fileName);
Console.WriteLine($"OuterXml equals contents: {fileContents == xd.OuterXml}");

var s = xd.DocumentElement["Assertion"]["Signature"];

var sx = new SignedXml(xd);
sx.LoadXml(s);

Console.WriteLine($"Signature verifies: {sx.CheckSignature()}");

Reference r = sx.SignedInfo.References[0] as Reference;

var base64 = xd.DocumentElement["Assertion"]["Signature"]["SignedInfo"]["Reference"]["DigestValue"].InnerText;
Console.WriteLine($"{BitConverter.ToString(Convert.FromBase64String(base64))} <- Read from XML");

Console.WriteLine($"{BitConverter.ToString(r.DigestValue)} <- DigestValue read by SignedXml");

xd.DocumentElement["Assertion"].RemoveChild(s);

var newSx = new SignedXml(xd);
newSx.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NTransformUrl; 
newSx.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA256Url;
newSx.SigningKey = RSA.Create();
var assertionId = xd.DocumentElement["Assertion"].GetAttribute("ID");
var newR = new Reference($"#{assertionId}");
newR.AddTransform(new XmlDsigEnvelopedSignatureTransform());
newR.AddTransform(new XmlDsigExcC14NTransform());
newR.DigestMethod = SignedXml.XmlDsigSHA256Url;

newSx.AddReference(newR);

newSx.ComputeSignature();

Console.WriteLine($"{BitConverter.ToString(newR.DigestValue)} <- DigestValue from new Signature");

Console.WriteLine($"New Signature Validates: {newSx.CheckSignature(newSx.SigningKey)}");

Console.WriteLine($"New signature, to validate references are configured the same:\n{newSx.GetXml().OuterXml}");

Output:

OuterXml equals contents: True
Signature verifies: False
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- Read from XML
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- DigestValue read by SignedXml
35-8C-77-8F-93-48-0E-E0-7E-39-76-BE-5F-8C-E1-27-44-EE-74-8A-E4-B3-EA-C1-6F-1F-3B-3B-70-43-3B-62 <- DigestValue from new Signature
New Signature Validates: True
New signature, to validate references are configured the same:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /><SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" /><Reference URI="#_09095fa6-55d9-4672-ade7-c1c696298200"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" /><Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /></Transforms><DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" /><DigestValue>NYx3j5NIDuB+OXa+X4zhJ0TudIrks+rBbx87O3BDO2I=</DigestValue></Reference></SignedInfo><SignatureValue>c10LMBgXvsSgA/XbEIvOui/1h3obqXOvGWz5NvMYqOoMKSqe6Zz6ijRVkKJcnqxWPOSJnYEghV99E0MepDMB55i1YH2mEtyMaD2GCA64qsCiozc9g+9Nu1HSB4bu6qK86wVI8/VA0UYblM8jyHQqxhgm8Uyj5YL2i+35DDjBFQs=</SignatureValue></Signature>
Shumiii commented 2 years ago

Many thanks @AndersAbel.

So, to get it working I had to comment out the line:

$"New Signature Validates: {newSx.CheckSignature(newSx.SigningKey)}".Dump();

I could not work out where to get this Dump extension method from but looks like I can still verify a lot without it so I continued.

So I ran the test console on the same file I gave you and I got this:

OuterXml equals contents: True
Signature verifies: False
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- Read from XML
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- DigestValue read by SignedXml
35-8C-77-8F-93-48-0E-E0-7E-39-76-BE-5F-8C-E1-27-44-EE-74-8A-E4-B3-EA-C1-6F-1F-3B-3B-70-43-3B-62 <- DigestValue from new Signature
New signature, to validate references are configured the same:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /><SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" /><Reference URI="#_09095fa6-55d9-4672-ade7-c1c696298200"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" /><Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /></Transforms><DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" /><DigestValue>NYx3j5NIDuB+OXa+X4zhJ0TudIrks+rBbx87O3BDO2I=</DigestValue></Reference></SignedInfo><SignatureValue>y07g9Zbp+9rEU5rNToTrA7tzAEBwBGE4eHC3egfPBfiso/zH55lT+09V2qMILK3MoL73uBxQyGPFymOYHfBw0nk9ILnvikSqHqVf0OtmPCXG4q+Jt33KDg1E1lEXWZX4wv6hE22HzeM5QGK4Y2uCYpDYO/cRydyOYu2BeHwV674=</SignatureValue></Signature>

I then ran it on one that I know works and I got this:

OuterXml equals contents: True
Signature verifies: True
54-87-7E-A6-9D-83-3D-65-EB-81-07-4A-C5-9B-48-B8-0E-BB-A2-22-ED-CB-2D-7C-24-A0-18-3D-4E-68-EE-B0 <- Read from XML
54-87-7E-A6-9D-83-3D-65-EB-81-07-4A-C5-9B-48-B8-0E-BB-A2-22-ED-CB-2D-7C-24-A0-18-3D-4E-68-EE-B0 <- DigestValue read by SignedXml

Thats as far as it got before it errored here:

Screenshot 2022-05-31 144648

I assume that the 'Signatire verifies' result is the one we are looking for? And the test is reflecting the real world scenario.

AndersAbel commented 2 years ago

The Dump() method is from Linqpad that I used to create the test. I thought I had cleaned them all up and changed them to Console.WriteLine, but appearantly not...

The reference constructor need to be updated when you run it on different data. It should be the ID of the Assertion node. I'll make an updated version that fixed the Dump() and reads the reference from the XML instead of hard coding it. I'll try to post it later today.

AndersAbel commented 2 years ago

@Shumiii I updated the code in the comment above, can you please make a new try?

The interesting part is really the DigestValue. It is a hash over the contents. The hash that is stored in the signature should of course be the same as the hash that we calculate ourselves of the data. If not, it usually indicates the data has been tampered with. But here it indicates that either side is doing an incorrect calculation.

Shumiii commented 2 years ago

@AndersAbel thanks, I've updated and re-ran.

Here is the output for the one we know fails:

OuterXml equals contents: True
Signature verifies: False
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- Read from XML
C8-E6-A9-F2-3E-ED-52-AC-2A-B0-4B-AA-32-05-DC-6F-30-F4-AF-2E-32-29-F2-A7-39-FD-53-B9-60-DB-AD-7C <- DigestValue read by SignedXml
35-8C-77-8F-93-48-0E-E0-7E-39-76-BE-5F-8C-E1-27-44-EE-74-8A-E4-B3-EA-C1-6F-1F-3B-3B-70-43-3B-62 <- DigestValue from new Signature
New Signature Validates: True
New signature, to validate references are configured the same:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /><SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" /><Reference URI="#_09095fa6-55d9-4672-ade7-c1c696298200"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" /><Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /></Transforms><DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" /><DigestValue>NYx3j5NIDuB+OXa+X4zhJ0TudIrks+rBbx87O3BDO2I=</DigestValue></Reference></SignedInfo><SignatureValue>dZJ12E89yYr9D13FOBFJmf49fFn9aojnUptaDHVk52lweqe0zVMNajNAKZ3C7PFWXPOBYYcBIBJpz923mruzJPSQAbtd4ptaxbh41qFGz50r+TY89OI0jVLtIdcNMbxhSOnrR4mUEeg3JPonjR9j6SbzeOJBw7YfGwS6thLa0Mw=</SignatureValue></Signature>

and here is the one that works:

OuterXml equals contents: True
Signature verifies: True
54-87-7E-A6-9D-83-3D-65-EB-81-07-4A-C5-9B-48-B8-0E-BB-A2-22-ED-CB-2D-7C-24-A0-18-3D-4E-68-EE-B0 <- Read from XML
54-87-7E-A6-9D-83-3D-65-EB-81-07-4A-C5-9B-48-B8-0E-BB-A2-22-ED-CB-2D-7C-24-A0-18-3D-4E-68-EE-B0 <- DigestValue read by SignedXml
54-87-7E-A6-9D-83-3D-65-EB-81-07-4A-C5-9B-48-B8-0E-BB-A2-22-ED-CB-2D-7C-24-A0-18-3D-4E-68-EE-B0 <- DigestValue from new Signature
New Signature Validates: True
New signature, to validate references are configured the same:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /><SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" /><Reference URI="#_050e9600-9a36-4b29-86a1-917b135a5f00"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" /><Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /></Transforms><DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" /><DigestValue>VId+pp2DPWXrgQdKxZtIuA67oiLtyy18JKAYPU5o7rA=</DigestValue></Reference></SignedInfo><SignatureValue>M72SVdDzi5vTxnJ3wY2LvbqEVeCuhitVkNpovS7fg/3iiqT2m6wKL78KQDZk/2frCIr8fv5jVOVgl8LvdXrt6Qv/PJ4K5GQw0BND6CySxDt9pLlbt/r/WCE8s0KMLLD4VJ6hTIgWi9ndK+Eul2h6KP1lzK9k3G3okeUfeh15SS0=</SignatureValue></Signature>
AndersAbel commented 2 years ago

@Shumiii Thank you very much. This confirms that it is the hash calculation that fails/gives a different value on the non-working example.

Now the next step is to figure out which one is calculating the wrong hash... I guess I'll have to find a third (not .NET) implementation and run the signature through it to see if it validates.

AndersAbel commented 2 years ago

Ok, I found this bug report that appears to be the same issue https://docs.microsoft.com/en-us/answers/questions/285425/saml-response-assertion-signature-validation-faile.html.

I also tried validating the SAML response I got from @Shumiii in another tool and it fails on signature validation. I'm leaning towards this being a bug on the AAD side.

AndersAbel commented 2 years ago

@samueldeva This is the Azure AD signature compatibility issue that we pinged you about on Twitter.

Shumiii commented 2 years ago

@AndersAbel yes I thought this might be an AAD bug too. I've uncovered a few bugs in Azure while working with their support teams so it is possible. A dev of mine, myself and you have all deduced that the signature is simply invalid. Sounding more and more like Azure is somehow not using the right key to sign (for certain accounts in my scenario). Feel more confident taking this to Azure support after your analysis thanks.

AndersAbel commented 2 years ago

@Shumiii It's actually not about the key. The signing is done in three steps.

  1. Calculate hash over the content to protect. This is the <Reference> and the hash value is the DigestValue.
  2. Calculate the hash of the previous hash together with some metadata about the signature - the <SignedInfo> node.
  3. Sign that second hash using the private key.

What the test program shows is that the failure is on step 1. The hash of the content to protect is not the same as the stored for the failing assertions when we use .NET's SignedXml to calculate it. I've read through the assertion, including in a hex editor to see if we have any special characters that would be the reason things are breaking - but no, nothing I can see.

One check that might be worth doing is to open one of the failing accounts in portal.azure.com and see if there are special characters in there that then somehow get's lost along the way? For example I noticed in the sample you sent that one of the attributes had the sequence space-space-dash-space as a separator in one attribute. Is that encoded in exactly that way in the portal?

Shumiii commented 2 years ago

Thanks @AndersAbel. Ok, so I'll pursue with Azure support to suggest they are hashing the content wrong for certain accounts. I cannot access the Azure AD to look at the screen but I did ask for some screenshots. However, why would we need that if we have the content in the SAMLResponse (like I posted above).

Is this not the content being hashed here? i.e. could I not be checking for illegal characters in here?

<Attribute Name="http://schemas.microsoft.com/identity/claims/tenantid">
                <AttributeValue>3401317f-bda5-49d1-8668-9d3caae9970a</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/identity/claims/objectidentifier">
                <AttributeValue>4198eebf-5e28-47b0-99c7-bea51084611c</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/identity/claims/identityprovider">
                <AttributeValue>https://sts.windows.net/3401317f-bda5-49d1-8668-9d3caae9970a/</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.microsoft.com/claims/authnmethodsreferences">
                <AttributeValue>http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password</AttributeValue>
                <AttributeValue>http://schemas.microsoft.com/claims/multipleauthn</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress">
                <AttributeValue>test@test.com</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/companyname">
                <AttributeValue>test.com</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/department">
                <AttributeValue>IT - Support</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/jobtitle">
                <AttributeValue>2nd Line Team Lead</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/lastname">
                <AttributeValue>Last</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/firstname">
                <AttributeValue>First</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/streetaddress">
                <AttributeValue>DIRFT 5  Dans Way</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/state">
                <AttributeValue>Northamptonshire</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/country">
                <AttributeValue>GB</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/officelocation">
                <AttributeValue>Daventry (Dist)</AttributeValue>
            </Attribute>
            <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/city">
                <AttributeValue>Crick</AttributeValue>
            </Attribute>
AndersAbel commented 2 years ago

@Shumiii I'm considering the case if there are some characters that have encoding issues

Consider the flow:

  1. Data is pulled from AAD database.
  2. XML is created in memory.
  3. XML is signed in memory
  4. The XML, with signature, is serialized into a string.
  5. We parse the XML into memory.
  6. We calculate the hash over the XML in memory representation as part of validation.

When it works, we obviously get the same hash value in step 3 and step 6. When it doesn't work, we get different values. What I'm thinking is that the problem could be in step 4. Then the hash calculated as part of step 3 would be correct for the in-memory representation of the XML. But if the serialization at step 4 does some substitutions that is actually someone altering the signed contents which gives different hashes. So reviewing the initial data in 1 and comparing to the data we see at 5 would tell us if there is an encoding issue that changes data somewhere in the flow.

AndersAbel commented 2 years ago

@Shumiii did you have any luck with getting in touch with the AAD team on this? I've tried through my contacts, but unfortunately no response :(

Shumiii commented 2 years ago

@AndersAbel I have not gone to Azure support yet. I have to go through the customer to use their Azure support subscription. What I had done was to get a screenshot in Azure AD and I have just received that. There are no suspicious values - I inputted the same values into an account over in my AAD and it worked fine. I will now get the customer to get Azure support to look at why their signature is not verifying for certain accounts through the Enterprise Saml Application. I'll give them a link to our discussion on here also. If you have anything else to add I'll gladly include. Thanks for your time on this.

AndersAbel commented 2 years ago

Thanks for the update. Based on what's been found so far I don't think that this is a bug in this library. I'm closing this as invalid but not locking the thread. If anyone has any new findings on the issue, please feel free to comment. If we can find out more about the cause and any indications that it indeed is a problem in this library I'll reopen.

Shumiii commented 2 years ago

@AndersAbel it has been resolved. Just like you said - a line break. Apologies for wasting your time. I did check the source data over and over and I could not see that line break. Even in the screenshot in AAD it was not there. Thank you for taking the time to debug with me.

AndersAbel commented 2 years ago

Was it a line break in the AAD attribute data? In that case the AAD serializer is handling this incorrect. Attribute values are technically allowed to contain line breaks, even though it's not a good idea.

Shumiii commented 2 years ago

@AndersAbel I'm not sure exactly how the line break got in there. I tried to replicate in Azure portal but the form does not let you enter in a new line. I suspect they had a local AD (with line break in data) which they have somehow got integrated or ported to AAD, and when AAD serializes they see the raw version with line break and when we serialize we see similar to Azure portal without line breaks.

ltanh01 commented 1 year ago

Hi @AndersAbel , @Shumiii You saved my life, thank you so much. I also have the same issue with this one.