AppGW using host

[reza8iucs]

Hi,

I was wondering what's the rational to pick the host name from backend ACA in the application gateway backend setting? isn't that against Microsoft's recommendation? Why not using the self-signed cert in the backend ACA app?

I spent a few hours to install this repo to see exactly how you set that up but the custom domain was not setup in ACA.

[thotheod]

Hi @reza8iucs. It is possible that some applications might fail if the reverse application proxy overrides the http host name, as it is described here. However there are also cases that you might intentionally want to do that, i.e. to avoid complexity of deploying split-horizon DNS etc.

If you need to preserve the host name, you can do it, and you have two options for that:

• option 1: Deploy everything as is (with Override with new host name set to true), and then use appGW's Rewrite HTTP headers feature to change the backend's response

• option 2: add a custom domain-with your own certificate in the Azure Container App. To do that you need:

  • a custom domain name. If you don't have any, you can buy an App Service domain
  • a certificate matching your custom domain. You can get a free certificate from let's encrypt. You can use Azure Key Vault to create a certificate request, and finally store the certificate securely. More details on that you can find in this blog post.

  Once you have those:

  • Add an A record on you custom Domain DNS zone pointing to the public IP of the application gateway
  • edit the AppGW httpListener to add your certificate matching your custom domain (in my case named mscaelab.com)

- Then change the https backend setting, set override with new host name to No

- Go to your ACA, go to Custom Domains and click Add a custom domain. There select the Bring your own certificate as shown below:   

- Then you need to do a domain validation. **IMPORTANT: ** You should not add the A record for the domain validation (as instructed in the portal see figure below), because this is the INTERNAL IP of the ACA Ingress, but for the same custom domain we need to resolve the Pip of the appGW. 

Once the domain is validated, and the certificate correct you get a green icon with "Secured" status in the custom domains list.

IMPORTANT NOTE: You should not change the app Gateway backendpool. The backend target should still point to the ACA default domain (i.e. ca-simple-hello.gentlebeach-ff5dee74.northeurope.azurecontainerapps.io) and not the custom domain. The Aca default domain can be resolved from the appGW because we have already (the LZA script does that) configured a private DNS zone. The ACA has a custom domain as hostname, but is not (and should NOT) be resolved from the external world.

I have tested the configuration described in option 2 above and you can check that it works here: https://mscaelab.com/ ( I will destroy the deployment in a few days though)

[reza8iucs]

Hi @thotheod,

Thanks for going the extra miles to set this up. Appreciate it!

When you set this up with your own custom cert and domain, did you change the AppGW backend setting with "Override with new host name" set to No?

if yes, can you do the exact same setup you did above with the new managed cert and instead of using the root domain use a subdomain like (https://**hello**.mscaelab.com/) with CNAME and TXT records and no A record? we have been banging our head against the wall for 2 days trying to set this up in ACA!

Thank you!

[thotheod]

Hi @reza8iucs . Yes I have changed the appGW backend setting with "Override with new host name" set to No. Even if you use the subdomain (i.e. hello.mscaelab.com) should not be different.

Bear in mind, that in order to validate your domain you need just to add the txt record. The CNAME or A record, must point to the public IP of the AppGW, not the private IP of the ACA. Ignore what is suggested in the portal UX when you try to Add a custom domain and certificate for ACA.

I have done the same for subdomain (testkv.mscaelab.com) and it works.

In ordetr to isolate and detect the problem, break down the process. Use the jumpbox VM to test that the custom domain works fine (connect through bastion, and add the internal IP of the ACA LB in the C:\Windows\System32\drivers\etc\hosts file. ) Check in the browser of the jumpbox if the custom domain works. If it works internally thenm something is wrong in the AppGW configuration

Do you get Green status here?

Does the broswer inside jump-box works with custom domain works? (local/internal IP set in hosts file):

[reza8iucs]

Thanks @thotheod. Much appreciate!

Two Questions: 1) In the backend pool , can we use the IP of the ACA environment instead of the FQDN of the app? Why not? 2) Also, did you try this with your new ACA's "Managed Certificate (preview)"? Or are you still using BYOD custom domain you used in your first reply?

P.S. I am really hoping that at some point you document and publish your setup in the ACA Docs. A lot of people can benefit from this, like the same one that currently exists for AppGW and APIM internal mode here. Just thinking loud here.

[thotheod]

Hi @reza8iucs, yes you can use the internal ACA IP, but in general it's better to use FQDNs. If you use FQDN you need to use the default ACA fqdn (not the custom domain) i.e. something like: ca-simple-hello.gentlebeach-ff5dee74.northeurope.azurecontainerapps.io

No you cannot use the "Managed certificate", because as you see in the first bullet here, your http ingress needs to be publicly available, which in the case of ACA LZA, is not.

[reza8iucs]

Hi @thotheod

When isolating the problem , I can confirm that I am able to hit the ACA app internally from the jumpbox by modifying the host file as per your instruction. So I think ACA custom domain is ok...mind you that i had to add the custom domain first and do the binding later as validation was not going through.

When hitting it through the app gateway,however, the only way I am able to get it to work is by applying the following in the backend setting otherwise AppGW gives me a 500 error because of an unhealthy probe. Did you use a custom health prob in your setup? I was hoping to get away with this whiteout having to set "Override with new host name" to true but it doesn't work for me. So, I am back to squar1 :)

P.S. I am not using the ALZ sample. I am using your hello world app deployed in an internal mode which is a similar setup to ALZ

[thotheod]

Hi @reza8iucs , Yes I Have a custom probe, and yes I have switched "Override with new host name" to No check screenshots of some AppGW settings below, I hope that helps:

backend pool (internal IP works as well instead of FQDN, I have tested both

Backend Settings (https)

httpListener:

Routing Rules (Listener)

Routing Rules (Backend Targets)

Custom probe (named: webProbe)

[reza8iucs]

Hi @thotheod

Thanks for including all the screenshots. You are simply amazing!

Modified the custom health prob Not to pick the host from the backend and specified the host in the custom health prob exactly like yours and solved the 500 error. However , now I am getting 502 bad gateway from AppGW unless I set the " "Override with new host name" to Yes and then specify "Pick host name from backend target"

we checked all of your screenshots and have exact same setup as yours but for us ""Override with new host name" to No doesnt work. It must be set to yes, and then the "Pick host name from backend target" must be seelcted.

Custom domain on the ACA app side works flawlessly as I am able to hit it from the jumbox with the host file change you proposed above. So I am either having a certificate issue as per the documentation here or a NSG/UDR is preventing the appgw to communicate with ACA with the host header not being overridden. I've never seen an issue like this before!

P.S. using ACA LB IP doesn't work for us. We must specify FQDN of the hello world ACA app otherwise we're getting the same 502 bad gateway error.

[reza8iucs]

Hi @thotheod

We managed to resolve the issue with the instructions you provided above. THANK YOU!

For anyone else who is getting 502 error in hitting their ACA app behind appGW. There are a number of root causes that you might be getting this issue which are documented here. Our issue was the certificate used in the ACA custom domain had the wrong subject (CN). Our certificate was issued tothe root domain (contoso.com ), but the ACA app was setup to use a subdomain (hello.constoso.com). In that case you either have to use a wildcard certificate (*.contoso.com) or a certificate that's specifically issued for that subdomain, in our case hello.contoso.com. We used the same certificate in the AppGW listener as well, as per @thotheod.

[thotheod]

Great!!! That's wonderful! Thank you very much for the update @reza8iucs, much appreciated.