Certificate chain reported as missing Intermediate certificate, throwing 502 error, with V2 Application Gateway only. V1 is fine.

[loopfish]

Certificate chain reported as missing Intermediate certificate, throwing 502 error, with V2 Application Gateway only. V1 is fine. This is will a full chain RapidSSL wildcard PFX certificate.

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 77a317f1-deef-4ba7-5839-f39ce3733655
• Version Independent ID: d85aa8fe-7270-d073-ea56-d1c0759383b8
• Content: Enabling end to end SSL on Azure Application Gateway
• Content Source: articles/application-gateway/ssl-overview.md
• Service: application-gateway
• GitHub Login: @amsriva
• Microsoft Alias: victorh

[shashishailaj]

@loopfish Thank you for your feedback . We will investigate and update the thread further.

[SubhashVasarapu-MSFT]

@loopfish, Try checking the intermediate certs on the backend cert. Here is an online tool for checking SSL certificates. Inccorect bundled cert would show something like one below: A Properly Bundled Cert would show something like the one below: Resolution: Certificate has to be properly bundled again including the Intermediate Certs. Please use a bundle of Root and Intermediate Certs.

[loopfish]

There is nothing wrong with the cert bundle. As I mentioned, it works absolutely fine with a v1 AG, and the Digicert SSL checker confirms all 3 parts of the cert are present and correct

[SubhashVasarapu-MSFT]

@loopfish, This requires a deeper investigation, so if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support. In this case, could you send an email to AzCommunity[at]Microsoft[dot]com referencing this thread and your subscription id?

Thank you for your cooperation on this matter and look forward to your reply.

[SubhashVasarapu-MSFT]

We are closing this issue for now. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.

[Kalmis]

I have a similar problem with normal Virtual Machines and Azure keyvault. I uploaded a bundle cert (server + intermediate) to Azure keyvault successfully. Then added the secret to a linux VM, so it is automatically fetched to /var/lib/waagent. However, the .crt file in the folder only includes the server certificate, i.e., intermediate certificate is no longer present.

No errors were thrown at any given point. Furthermore, if I download the cert in PFX/PEM format from Azure portal UI, the intermediate cert is still present in this file.

Why is the intermediate cert removed from the .crt file?

[loopfish]

Unfortunately Microsoft don't seem to want to know, on this occasion. I spent a lot of time with support and escalation demonstrating there was nothing wrong with my cert bundle, but they insisted it was. Given up using the product due to this.

[Kalmis]

@SubhashVasarapu-MSFT Any comment on this?

[HQJaTu]

Any comment on this?

I just upgraded an AppGw V1 into V2 and looks like I need to roll it back. Intermediate CA certificate needs to be delivered in the TLS handshake. I'm even using the same certificate .pfx file than with V1 and it worked there. With V2, no such joy.

[georgegil]

Got the same issue, the intermediate certs are not being sent during the TLS handshake

[loopfish]

@shashishailaj please could you re-open this issue for investigation?

[HQJaTu]

I got my AppGw V2 working. The clues for solution are scattered, but they are out there.

What Azure fails to properly document is the new requirement of PFX-file to contain the intermediate too. However, my case was bit complex. I had bound my AppGw certificate into Azure Key Vault. However, you cannot have a Key Vault Certificate with intermediate (or entire chain).

So, I detached the Key Vault association and wrote some PowerShell scripting to export a .Net Security.Cryptography.X509Certificates.X509Certificate2Collection and Set-AzureRmApplicationGatewaySslCertificate that into AppGw. Works great!

[georgegil]

One would imagine using a key vault is encouraged. Indeed that is what I am using.

[HQJaTu]

Yes, using a Key Vault is encouraged and that's what I'm still doing.

The solution is obvious, store the Base64-encoded PFX-file into a secret. The code I'm using is adapted from https://www.rahulpnath.com/blog/pfx-certificate-in-azure-key-vault/

[jtaubensee]

I experienced the same issues that others above have been facing. In particular, we only had the SSL validation errors for certain devices like Android and Python on Windows. But we also had the same symptoms that were mentioned here: https://github.com/MicrosoftDocs/azure-docs/issues/35298#issuecomment-514918980

After contacting Azure support, we did not have any luck solving things, but we eventually just exported the certificate with all intermediary certificates in the chain using the Certificate Manager MMC Snap In.

While this was a very easy problem to solve, it was not particularly discoverable.

[jaffyali]

I experienced the same issues that others above have been facing. In particular, we only had the SSL validation errors for certain devices like Android and Python on Windows. But we also had the same symptoms that were mentioned here: #35298 (comment)

After contacting Azure support, we did not have any luck solving things, but we eventually just exported the certificate with all intermediary certificates in the chain using the Certificate Manager MMC Snap In.

While this was a very easy problem to solve, it was not particularly discoverable.

i have implemented your solution and it worked for me, but now whrn i test my ssl it shows "Incorrect order, Contains anchor"

--

[RoboJaTu]

i have implemented your solution and it worked for me, but now whrn i test my ssl it shows "Incorrect order, Contains anchor"

You're not supposed to have the anchor cert in your AppGw. Only intermediate and site cert needed.

Having the anchor there isn't an error. The client is supposed to ignore it. This obviously will vary.

[jaffyali]

i have implemented your solution and it worked for me, but now whrn i test my ssl it shows "Incorrect order, Contains anchor"

You're not supposed to have the anchor cert in your AppGw. Only intermediate and site cert needed.

Having the anchor there isn't an error. The client is supposed to ignore it. This obviously will vary.

i have used .pfx file on appGw . now i am having

Chain issues | Incorrect order, Contains anchor.what is this.

-- | --

[dfendit]

+1 having the same issue here after testing it through SSLLABS on App Gw v2. Any solution for this? Using Key Vault, but if you try to import your PFX into a secret, it states that the functionality is deprecated.

[HQJaTu]

Azure Key Vault cannot import PKCS#12 (or a PFX if you will) into a secret.

What I do is import the bytes of the PKCS#12. See my comment above for details.

Also, please investigate the new approach of AppGw Key Vault integration. Docs are at https://docs.microsoft.com/en-us/azure/application-gateway/key-vault-certs.

[dfendit]

Doesn't work and why should I bother to follow a workaround. This should be working as expected. I cannot believe that MS is not solving this issue/bug/security flaw.

[dfendit]

i have implemented your solution and it worked for me, but now whrn i test my ssl it shows "Incorrect order, Contains anchor"

You're not supposed to have the anchor cert in your AppGw. Only intermediate and site cert needed. Having the anchor there isn't an error. The client is supposed to ignore it. This obviously will vary.

i have used .pfx file on appGw . now i am having

Chain issues | Incorrect order, Contains anchor.what is this.

-- | --

I have the same error.

[dfendit]

Please reopen this ticket

[RikGr]

Also when we use the Key Vault method to select the SSL certificate in the Listener, the order is not correct:

Please reopen this issue.

[ohernand99]

So, here is the straight forward resolution, two clicks away, It worked for me, should work for you as well. This lines apply for a real SSL certificated (not self-signed) issued by an non integrated CA to Azure (there are only two CAs integrated, the remaining 99% CAs are left in the wind). In other words, this apply if you are installing on Azures Vault a certificate issued by GoDaddy e.g.

My problem started with Azure's V2 Application Gateway, after creating a Vault, an Identity manger, creating a Certificate Request in the Vault, merging the Signed Request, configuring in the Application Gateway my listeners, rules, and HTTP settings everything seemed in place. In fact, requesting my https site from any Browser looked great, the problem started when trying to access an application on this URL through and Android App. The first red flag prior to that was a request through Postman that showed a warning saying SSL Error: Unable to verify the first certificate

Everything went south from that point forward. After reading all the contributions on this forum I came up with an idea, it's so simple but it's not documented anywhere.

The problem can be easily seen through and SSL check, you can find a nice tool to know if you are going through the same:

https://www.sslshopper.com/ssl-checker.html

As you can see, an error comes up during the check saying The Certificate is not trusted in all web browsers. You may need to install an Intermediate/chain certificate to link it to a trusted root certificate ...

So, here is the resolution, no open SSL, no weird Azure CLI commands, just a couple of clicks away...

1.- Create a Certificate Request using Azures Vault, nothing funny to be configured on the Advance Policy Configuration, although I strongly recommend a Key Size of 4096.

2.- After Creating the Request, click on your cert, which at this point will be listed on the "In progress, failed or cancelled" section.

3.- Click on Certificate Operation and then Download CSR option.

4.- Go to your CA authority and issue your cert as regularly, wait for it to be completed.

5.- The CA will issue commonly two files, a certificate.crt and a ca_bundle.crt, discard the bundle we only need the ceretificate.crt

-- On a regular scenario you would take ce certificate.crt and upload it to the Vault by accessing the Certificate Operation option and Clicking on the Merge Signed Request. I mean, it's obvious and it work with most web browsers but in truth the SSL chain of that cert will be incomplete and that's what we are addressing here --

6.- Download the Cert to your computer (we are using Windows OS for this operation). Right click on it and click on Install Certificate, place the certificate some store you can find it, I used the Personal Store as it is the first one in the list, next, next, next and finish the operation.

7.- Open your Control Panel and in the search box type "certificates" and locate "Manage User Certificates" (I don't know why if I open crtmgr from the Windows button, the same MCC will appear but a limited list of certificates)

8.- Go to the Store you imported it and right click it, note that the certificate you are looking for is named as the CN (Common Name you used for requesting it), select the Export option and here comes the trick...

9.- Select in the options "Cryptographic Messages Syntax Standard - PKCS#7 Certificates (.p7b) and SELECT THE "INCLUDE ALL CERTIFICATES IN THE CERTIFICATION PATH IF POSSIBLE"

10.- Browse for a location to save this file, preferably on the same place you downloaded the certificate.crt file just for keeping an order.

11.- Go to the Vault, select the certificate which at this point is located on the "In progress, failed or cancelled", go to the Certificate Operation option, and in the Merge Signed Request UPLOAD the .p7b file you created during the export process.

12.- If you follow all the steps correctly the Completed Green check will appear.

13.- And you are all set!!! Finish the configuration of the Application Gateway, your listeners, rules and HTTP setings and select the Certificate from the Vault as regularly.

14.- After you publish your service you can go to https://www.sslshopper.com/ssl-checker.html again and perform a check, you will see this time the certificate will show a correct SSL chain.

Hope you find this useful, I struggle for a complete week to figure it out as there is no proper documentation in Azure,

[HQJaTu]

So, here is the straight forward resolution, two clicks away ...

This ticket was opened over two years ago. Two years is a long time in cloud world. During that time one may expect things to change, improve even. In this particular case improvements were made and at the time of writing, what you suggest works.

[ohernand99]

Here is an update to my previous post. As reported on some forums, merging a p7b cert has become unavailable according to MIcrosoft as reported on this post from a Microsoft specialist https://docs.microsoft.com/en-us/answers/questions/713593/got-the-error-while-merging-the-certificate-in-azu.html. on Feb 2022.

Unfortunately the problem of the broken SSL chain if you decide to merge the .cer downloaded from your CA.

The new word around is as follows: In the case of many CA they will deliver to you a .cer along with a bundle.cer file containing the intermediate certificate to complete the chain.

1) Open the Bundle.cer file in a notepad 2) Open the certificate that was issued to you, the .cer file in a notepad 3) Copy the .cer file content INCLUDING THE HEADER AND FOOTER at the end of the content of the bundle.cer file 4) Save it into a new .cer file 5) Merge it using the Merge Signed Request option in Azure.

In my case, I used goDaddy to issue my certificate, the bundle contained several certificates, so at the end the content of my new file looked like this:

-----BEGIN CERTIFICATE----- fhdjskfhdjskfhdsjkalfhdsjakfhdsjkflhsdajkhjklhjklhjk -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- fhjdkasfhdsjkalfhdsjaklfhdsajklfhdsjkflsdhafjkshdajk rw== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- djfkhdsljkfhdsjkfhsdajkfdhsjkafhdsjakfhdsjakfhdsjkl -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- dhfdasfdhsjkfdhsjakfhdsfjkadshfjkldsahfjklsadhjkhjkl -----END CERTIFICATE-----

against all odds, it worked!!!

Hope you find this information useful.

[desmphil]

Great Article. I tried your recipe above, merging the Cert.CER, the intermediate.CER and the root.CER in a single file using the Headers. Merged Signed request.

I though at first that it wasn't working but. In order to refresh the Application Gateway Certificate from the Key Vaullt i needed to force the things up and force the Refresh of the certificate on the listerners. I should have been patient..

SSL Labs changed from B to A+
However i still get this issue:  Chain issues | Incorrect order, Contains anchor
I will try to switch the Root and Intermediate order in the new cer

Switching the Order did fix the error above.

According to SSL Labs, only the intermediate is needed. Error Contains anchor: It means that you have added Intermediate as well as Root CA, when you only need the Intermediate as the client will already have Root CA (will be already trusted by browser in browser certificate store).

[Theyssens]

https://www.windowstechupdates.com/azure-application-gateway-chain-issues-incorrect-order-contains-anchor/

[varnav]

I have same cert + intermediate bundle, and I have same AppGw's - one in East 1 and another in West 1 regions. They behave differently with same PFX. East one works fine, returns cert as is. West one "contains anchor".

[samrueby]

This is still an issue with application gateway v2 when using key vault certificates.

[razx-x]

I selected include all certificates and the issue is resolved in my case I was directly uploading certificate to WAF V2 instead of using key vaults certificates.

[mendoncaftw]

Still an issue in 2024

[hjchin]

It is still an issue in Aug 2024