projectkudu / kudu

Kudu is the engine behind git/hg deployments, WebJobs, and various other features in Azure Web Sites. It can also run outside of Azure.
Apache License 2.0
3.13k stars 652 forks source link

Kudu.Service running on k8sebuild-service for azure hci stack // aks hybrid // app-service // behind Proxy #3493

Closed chris8205 closed 7 months ago

chris8205 commented 1 year ago

Repro steps.

trying to deploy a functionapp on Azure Stack HCI ( Hybrid AKS ) The system is behind a proxy. i can see the proxy is published in the environment of the k8sebuild-service pod HTTP_PROXY, HTTPS_PROXY and NO_PROXY are properly set by "init"

now the kudu tries to get secrets from the k8s (internal connection) but detects that proxy is configured, and tries to open the request to the K8s Service "Default"."Kubernetes" on Port 443 over https.

but, while using a proxy the request is directly send over the proxy instead trying to handle the request internally without a proxy.

seems that the NO_PROXY=10.0.0.0/8,... ... ... is ignored. The LAN proxy can't never reach the internal k8s service in our environment 10.96.0.1

because running in a timeout the result of the "ReadNamespacedSecretWithHttpMessagesAsync" is wrong handled..

The log/error given by the failure.

Exception Message : Operation returned an invalid status code 'ServiceUnavailable' ││ Exception StackTrace : at k8s.Kubernetes.ReadNamespacedSecretWithHttpMessagesAsync(String name, String namespaceParameter, Nullable││ 1 exact, Nullable1 export, String pretty, Dictionary2 customHeaders, CancellationToken cancellationToken) ││ at k8s.KubernetesExtensions.ReadNamespacedSecretAsync(IKubernetes operations, String name, String namespaceParameter, Nullable1 exa ││ ct, Nullable1 export, String pretty, CancellationToken cancellationToken) ││ at k8s.KubernetesExtensions.ReadNamespacedSecret(IKubernetes operations, String name, String namespaceParameter, Nullable1 exact, N ││ ullable1 export, String pretty) ││ at Kudu.Core.K8SE.K8SEDeploymentHelper.<>c__DisplayClass15_0.<UpdateContextWithAppSettings>b__0() in /tmp/KuduLite/Kudu.Core/K8SE/K8 ││ SEDeploymentHelper.cs:line 227 ││ at Microsoft.Rest.TransientFaultHandling.RetryPolicy.<>c__DisplayClass16_0.<ExecuteAction>b__0() ││ at Microsoft.Rest.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func) ││ at Microsoft.Rest.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action) ││ at Kudu.Core.K8SE.K8SEDeploymentHelper.UpdateContextWithAppSettings(IKubernetes client, HttpContext context) in /tmp/KuduLite/Kudu.C ││ ore/K8SE/K8SEDeploymentHelper.cs:line 243 ││ at Kudu.Services.Web.KubeMiddleware.Invoke(HttpContext context, IEnvironment environment, IServerConfiguration serverConfig) in /tmp ││ /KuduLite/Kudu.Services.Web/KubeMiddleware.cs:line 144 ││ at Kudu.Services.Web.Tracing.TraceMiddleware.Invoke(HttpContext context) in /tmp/KuduLite/Kudu.Services.Web/Tracing/TraceMiddleware. ││ cs:line 64

Hope it's not to strange written....

EDIT:

found https://github.com/projectkudu/kudu/blob/93f95daef75893e9cafce20bdc7e8946decf06c7/Kudu.Core/Scripts/dnvm.ps1#L550 this file what is pulling $env:http_proxy variable... after removing http(s)_proxy env-params, the container was able to fetch the 10.x.x.x:443 internal k8s service...

maybe if this is the correct code part, if added the BypassList to the "WebProxy" can help ?

jvano commented 7 months ago

Hi

If the problem persists and is related to running it on Azure App Service, please open a support incident in Azure: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

This way we can better track and assist you on this case

Thanks,

Joaquin Vano Azure App Service