Hostname Header Propogation with Azure App Gateway and BtK

[xxthegonzxx]

Hi,

I have an issue when I try to run Bridge to Kubernetes with isolation enabled. The BtK extension works great and set's up a clone of my Pod and all the services running in my "dev" namespace. It also sets up ingress clones with generated hostnames that point back to the original services running in my "dev" namespace.

The problem that I have is that I am using App Gateway Ingress Controller (AGIC) with hostname annotation, which redirects hostnames like 'my-app.example.com` to my internal backend service and when Bridge to Kubernetes creates ingress for redirection to existing pods in the "dev" namespace it doesn't work because of the AGIC annotation specifically expecting back the hostname 'my-app.example.com' in the ingress.yaml annotation: https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/annotations.md#backend-hostname.

Since the BtK extension creates these generated names for ingress how would I handle that type of hostname redirection? I'm wondering if there is a better way to handle this that I'm not thinking about? Any help is appreciated. Great work on the extension BTW!

[pragyamehta]

Hi @xxthegonzxx Thanks for trying out Bridge to Kubernetes and reporting this issue. You are right, as of today, we do not support this annotation properly. But you can still hit your breakpoints by sending a request to the original ingress URL and setting the header - "kubernetes-route-as": "routeAs-header-value". The routeAs-header-value is the value generated specifically for you when you configure Bridge to Kubernetes with isolation enabled. You can view that value in your launch.json. (Please make sure you propogate the kubernetes-route-as header in all your microservices!)

Hope this unblocks you for now. I have also created a work item on us to handle this scenario properly. Thanks for reaching out to us!

[xxthegonzxx]

Thanks for the quick reply @pragyamehta! Can you give a little more guidance with how to set the header? Do you mean literally in the browser to set the header in the request? This might work with some of our API's but I don't think it will work with our front-end applications. My concern here is that it does not scale well. Especially since we have several developers and we will have to hardcode that routeAs-header-value several times in our micro-services; one for each developer! Please let me know if that is the case because if so I will have to find another alternative to BtK in the meantime until this is supported. Thank you for your time.

[pragyamehta]

Hi @xxthegonzxx , I apologize for the confusion, you do not need to hard code the value of the header. That would indeed be non-scalable. Each of the microservices need to read the "kubernetes-route-as" header on the incoming request and set this header with its incoming value on the outgoing response. This ensures that whatever the value was set to, with the incoming request gets forwarded throughout the system and ensures that routing for your request works properly i.e. all requests prefixed with your routeAs value will reach your machine for debugging purposes. Other requests that target the stable version of the application (i.e. ingress domains without the kubernetes-route-as header set) would not come to your machine for debugging.

Let me know if this makes sense!

Edit: For setting the header while making the request, you would have to use something like fiddler/postman to construct the header in the request. This is not an ideal experience, which is why I have logged a bug on our team to get this resolved. I will update this thread once we release the fix. Thanks for your patience!

[pragyamehta]

An example of propogating the header is in our sample of BikesharingApp - in this file HttpClientExtensions.cs

[greenie-msft]

We've captured supporting AGIC as a work-item in our backlog. We've also provided a workaround and guidance on how to properly propagate the isolation header between services.

Will keep this thread open until we have an update on supporting AGIC to share.

[davejhahn]

We've captured supporting AGIC as a work-item in our backlog. We've also provided a workaround and guidance on how to properly propagate the isolation header between services.

Will keep this thread open until we have an update on supporting AGIC to share.

@greenie-msft is the guidance you mention regarding isolation headers in reference to AGIC? Is that just what is posted in this same thread or was there other documentation?

We have had 2 major issues with Azure DevSpaces in the last 3 weeks--both on Microsoft's end, where it wasn't available in new or existing clusters. We've lost over 3 days worth of development time. We want to move to Bridge, but the AGIC support is the one thing holding us back. Not only is it causing us problems, AzDS isn't supported past 1.18.14, so we can't upgrade Kubernetes anymore either until we get rid of AZDS.

I read through the above, and I'm not sure I see that as enough details to work through the AGIC issues. Is it possible to provide some more detail? I think it might help others as well.

Thanks,

Dave