microsoft / mindaro

Bridge to Kubernetes - for Visual Studio and Visual Studio Code
MIT License
307 stars 106 forks source link

Failed to refresh the kubeconfig token: Operation returned an invalid status code 'BadGateway' #200

Open hirannair opened 3 years ago

hirannair commented 3 years ago

Describe the bug Im getting the below error when trying to configure Bridge to kubernetes

Failed to refresh the kubeconfig token: Operation returned an invalid status code 'BadGateway'

To Reproduce Install bridge to kubernetes extension in vscode and try to configure it

Ours is a private kubernetes cluster that runs in our datacentres, and we manage it.

Not sure why the extension is trying to refresh the token as we have a separate login process to get a new token.Our k8s token is a long lived token,so its not required to refresh it.

**Expected behavior** A clear and concise description of what you expected to happen. **Logs** ``` 2021-07-09T03:10:27.248Z | Common Extension Root | TRACE | Event: kubectl-client-command {"args":"config view --minify -o json","remoteName":"dev-container"} 2021-07-09T03:10:27.390Z | Common Extension Root | TRACE | Event: kubectl-client-command-success {"args":"config view --minify -o json","remoteName":"dev-container"} 2021-07-09T03:10:30.260Z | Common Extension Root | ERROR | Error: cli-client-refresh-credentials-output-error {"remoteName":"dev-container"} SyntaxError: Unexpected token O in JSON at position 0\n at JSON.pars e ()\n at Array. (/root/.vscode-server/extensions/mindaro.mindaro-1.0.120210709/dist/extension.js:3:43482)\n at t.EventSource.trigger (/root/.vscode-server/extensions/mindaro.mindaro-1.0.120210709/dist /extension.js:1:56669)\n at Socket. (/root/.vscode-server/extensions/mindaro.mindaro-1.0.120210709/dist/extension.js:3:143963)\n at Socket.emit (events.js:315:20)\n at addChunk (internal/streams/readable.js:30 9:12)\n at readableAddChunk (internal/streams/readable.js:284:9)\n at Socket.Readable.push (internal/streams/readable.js:223:10)\n at Pipe.onStreamRead (internal/stream_base_commons.js:188:23) 2021-07-09T03:10:32.308Z | Common Extension Root | ERROR | Error: cli-client-refresh-credentials-error {"remoteName":"dev-container"} Error: Operation returned an invalid status code 'BadGateway'\n\n at ChildProc ess. (/root/.vscode-server/extensions/mindaro.mindaro-1.0.120210709/dist/extension.js:3:144142)\n at ChildProcess.emit (events.js:315:20)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12) 2021-07-09T03:10:32.311Z | Connect (docker-dev/db6a | ERROR | Error: kubeconfig-credentials-manager-refresh-credentials-error {"remoteName":"dev-container"} Error: Operation returned an invalid status code 'BadGatewa y'\n\n at ChildProcess. (/root/.vscode-server/extensions/mindaro.mindaro-1.0.120210709/dist/extension.js:3:144142)\n at ChildProcess.emit (events.js:315:20)\n at Process.ChildProcess._handle.onexit (internal/chi ld_process.js:277:12) 2021-07-09T03:10:32.312Z | Connect (docker-dev/db6a | TRACE | Event: kubeconfig-credentials-manager-refresh-credentials-perf {"success":false,"refreshRequired":false,"durationMs":4921,"remoteName":"dev-container"} 2021-07-09T03:10:32.315Z | Connect (docker-dev/db6a | TRACE | Event: connect-wizard-stop {"wizardReason":"ConfigurationCommand","type":"service","isWizardComplete":false,"isResourceNameSet":"false","isIsolateAsSet":"false","isTarg etClusterSet":"false","isTargetNamespaceSet":"false","isCreatingNewLaunchConfiguration":false,"remoteName":"dev-container"} ``` **Environment Details** Client used (VS Code/Visual Studio): VS Code Client's version: Operating System: **Additional context** Add any other outputs from the clients or context you would like to share.
daniv-msft commented 3 years ago

Thanks @hirannair for opening this issue. When we "refresh credentials", we actually delegate that operation to kubectl:

In the case you describe, it sounds like we're having troubles with the reply we get from kubectl.

Would it be possible for you to run kubectl get pods -n <YOUR_NAMESPACE> for the same cluster you use with Bridge to Kubernetes, and let me know what response you get?

hirannair commented 3 years ago

Thanks @daniv-msft for that quick response.

$ kubectl get pods
NAME                                            READY   STATUS    RESTARTS   AGE
fin-nginx-ingress-controller-58556b9589-ccj9b   1/1     Running   0          61d
fin-nginx-ingress-controller-58556b9589-qsb8m   1/1     Running   0          61d
fin-redis-cache-master-0                        1/1     Running   0          61d
fin-redis-cache-slave-0                         1/1     Running   4          61d
fin-redis-cache-slave-1                         1/1     Running   2          51d
routingmanager-deployment-5d75f8cb77-ptg2r      1/1     Running   0          23h
hirannair commented 3 years ago

Im able to get the list of pods through kubectl command line and kubernetes extension in vscode

daniv-msft commented 3 years ago

Thanks @hirannair for your reply. I investigated more this issue, and it's possible that it comes from the version of the C# KubernetesClient library we use. We're using a version that isn't the latest, and it's possible this version is the one returning the BadGateway error in your case.

We're working on integrating with their latest version to determine if it helps with your case. Would you be willing to try beta bits of our extension including this change? I can provide full instructions once the fix is available (we're trying to get it done by tomorrow).

hirannair commented 3 years ago

@daniv-msft Thanks for the response.Sure,Happy to test the beta version of the extension if you can provide the instructions on how to get it running

daniv-msft commented 3 years ago

(+ @amsoedal who worked on increasing our dependency's version) @hirannair Thanks! We're working to provide this to you tomorrow.

amsoedal commented 3 years ago

Hi @hirannair, I've posted the steps here (https://github.com/microsoft/mindaro/issues/174#issuecomment-879313528) on how to test beta bits. Please let us know how it goes for you! :)

hirannair commented 3 years ago

Thanks @amsoedal .Still getting the same error

below are the logs


2021-07-14T00:57:50.266Z | Common Extension Root    | TRACE | query-expfeature <json>{"ABExp.queriedFeature":"vscode.mindaroBinariesVersion-1.0.20210702","remoteName":"dev-container"}</json>
2021-07-14T00:57:50.266Z | Common Extension Root    | TRACE | Call to ExP returned version '1.0.20210708.15' <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.266Z | Common Extension Root    | TRACE | Resolved expected CLI version '1.0.20210712.3' <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.286Z | Common Extension Root    | TRACE | Event: binaries-utility-version-v1 <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.300Z | Common Extension Root    | TRACE | Trying to initialize the workspace folder docker-dev for Connect <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.301Z | Common Extension Root    | TRACE | Connect initialization started on docker-dev <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.305Z | Common Extension Root    | TRACE | Event: connect-initialization-success <json>{"workspacesCommonId":"22404a8d-400c-7e38-a99b-abaa1cc52737","remoteName":"dev-container"}</json>
2021-07-14T00:57:50.316Z | Common Extension Root    | TRACE | Making sure that the CLI is present locally, by downloading it if needed <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.345Z | Common Extension Root    | TRACE | Making sure that the CLI is present locally, by downloading it if needed <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.368Z | Common Extension Root    | WARNG | Binaries are not present locally: downloading them <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:50.387Z | Common Extension Root    | TRACE | Resolved versionUrl 'https://mindarostaging.blob.core.windows.net/zip/1.0.20210712.3/lks.json' <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:57:51.734Z | Common Extension Root    | TRACE | Event: cli-versions-client-get-download-info-success <json>{"availableVersion":"1.0.20210712.3","expectedVersion":"1.0.20210712.3","remoteName":"dev-container"}</json>
2021-07-14T00:57:51.735Z | Common Extension Root    | TRACE | Event: cli-version-client-get-download-info-status <json>{"binariesVersionsDownloadTimeInMilliseconds":1361,"binariesVersionsDownloadSucceeded":true,"remoteName":"dev-contain
er"}</json>
2021-07-14T00:57:51.736Z | Common Extension Root    | TRACE | Event: binaries-utility-download-start <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:05.247Z | Common Extension Root    | TRACE | Event: configure-command-triggered <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:05.250Z | Connect (docker-dev/444d | TRACE | Event: connect-wizard-start <json>{"wizardReason":"ConfigurationCommand","type":"service","remoteName":"dev-container"}</json>
2021-07-14T00:58:05.254Z | Common Extension Root    | WARNG | Event: binaries-utility-try-get-binaries-error <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:05.254Z | Common Extension Root    | WARNG | Event: binaries-utility-try-get-binaries-error <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:05.255Z | Connect (docker-dev/444d | TRACE | Event: connect-wizard-stop <json>{"wizardReason":"ConfigurationCommand","type":"service","isWizardComplete":false,"isResourceNameSet":"false","isIsolateAsSet":"false","isTarg
etClusterSet":"false","isTargetNamespaceSet":"false","isCreatingNewLaunchConfiguration":false,"remoteName":"dev-container"}</json>
2021-07-14T00:58:18.043Z | Common Extension Root    | TRACE | Making sure that the CLI is present locally, by downloading it if needed <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:46.835Z | Common Extension Root    | TRACE | Event: binaries-utility-download-success <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:46.836Z | Common Extension Root    | TRACE | Event: binaries-utility-overall-download-status <json>{"binariesDownloadTimeInMilliseconds":55100,"binariesDownloadSucceeded":true,"remoteName":"dev-container"}</json>
2021-07-14T00:58:50.243Z | Common Extension Root    | TRACE | Event: cli-client-get-version-success <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:50.244Z | Common Extension Root    | TRACE | Found local CLI version: '1.0.20210712.3'. Expected version: '1.0.20210712.3' <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:50.244Z | Common Extension Root    | TRACE | Local CLI has version number '1.0.20210712.3', which is equal to the required '1.0.20210712.3' <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:50.245Z | Common Extension Root    | TRACE | Event: binaries-utility-ensure-cli-success <json>{"isUsingLocalBinaries":"false","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.257Z | Common Extension Root    | TRACE | Event: kubectl-client-command <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.279Z | Common Extension Root    | TRACE | Event: kubectl-client-command <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.329Z | Common Extension Root    | TRACE | Event: kubectl-client-command <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.642Z | Common Extension Root    | TRACE | Event: kubectl-client-command-success <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.644Z | Common Extension Root    | TRACE | Event: kubectl-client-command <json>{"args":"config view -o jsonpath={.clusters[*].cluster.server}","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.680Z | Common Extension Root    | TRACE | Event: kubectl-client-command-success <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.687Z | Common Extension Root    | TRACE | Event: activation <json>{"workspacesCommonId":"22404a8d-400c-7e38-a99b-abaa1cc52737","workspaceFoldersCount":"1","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.687Z | Common Extension Root    | TRACE | Extension activated successfully <json>{"remoteName":"dev-container"}</json>
2021-07-14T00:58:50.768Z | Common Extension Root    | TRACE | Event: kubectl-client-command-success <json>{"args":"config view --minify -o json","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.769Z | Common Extension Root    | TRACE | Event: kubectl-client-command <json>{"args":"config view -o jsonpath={.clusters[*].cluster.server}","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.904Z | Common Extension Root    | TRACE | Event: kubectl-client-command-success <json>{"args":"config view -o jsonpath={.clusters[*].cluster.server}","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.904Z | Common Extension Root    | TRACE | Event: kubernetes-panel-customizer-supported-fqdn-evaluation <json>{"currentFqdnDomain":"co.nz","clustersCount":4,"fqdnDomains":"co.nz,co.nz,co.nz,co.nz","remoteName":"dev-co
ntainer"}</json>
2021-07-14T00:58:50.992Z | Common Extension Root    | TRACE | Event: kubectl-client-command-success <json>{"args":"config view -o jsonpath={.clusters[*].cluster.server}","remoteName":"dev-container"}</json>
2021-07-14T00:58:50.993Z | Common Extension Root    | TRACE | Event: kubernetes-panel-customizer-supported-fqdn-evaluation <json>{"currentFqdnDomain":"co.nz","clustersCount":4,"fqdnDomains":"co.nz,co.nz,co.nz,co.nz","remoteName":"dev-co
ntainer"}</json>
2021-07-14T00:59:51.954Z | Common Extension Root    | TRACE | Extension deactivation <json>{"remoteName":"dev-container"}</json>
daniv-msft commented 3 years ago

Thanks @hirannair for giving it a try! I confirm you're running the right version of our binaries, so unfortunately it looks like this issue wasn't fixed in the latest version of our dependency. I will open a Github issue on their side to report that and see how we can proceed from here. Before I do, and to make sure I have enough information to open the issue on their repository, could you please reproduce the bug and attach the logs from this folder?

I'm interested in the latest bridge-library-*.txt and bridge-mindarocli-*.txt files, that should show the exact exception we get from the C# KubernetesClient. Thanks for your help!

hirannair commented 3 years ago

@daniv-msft Thanks for that quick response.Below are the logs

$ cat bridge-mindarocli-2021-07-14-01-14-13-1344.txt
2021-07-14T01:14:13.9329823Z | MindaroCli | TRACE | Event: Command.Start <json>{"properties":{"arguments":"refresh-credentials --namespace fin-acceptance","isRoutingEnabled":"false"},"metrics":null}</json>\nOperation context: <json>{"cl
ientRequestId":null,"correlationId":"b0464677-5cce-4795-9b1f-912f86f4ccc81626224689906:4559d60919c2","requestId":null,"userSubscriptionId":null,"startTime":"2021-07-14T01:14:13.5524405Z","userAgent":"VSCode/1.0.120210713","requestHttpMe
thod":null,"requestUri":null,"version":"1.0.20210712.3","requestHeaders":{},"loggingProperties":{"applicationName":"MindaroCli","deviceOperatingSystem":"Linux 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021","framework":".NET Core
3.1.9","macAddressHash":"3424b72ee104e99be183dc60a7b751a058f3ff95747d04922fb59889f7e032b1","processId":1344,"targetEnvironment":"Staging","commandId":"4559d60919c2"}}</json>
2021-07-14T01:14:14.0424937Z | MindaroCli | TRACE | Running Microsoft.BridgeToKubernetes.Exe.Commands.Connect.RefreshCredentialsCommand...
2021-07-14T01:14:14.0459617Z | MindaroCli | TRACE | Initializing progress reporter...
2021-07-14T01:14:15.2041204Z | MindaroCli | ERROR | Dependency: List Contexts <json>{"target":null,"success":false,"duration":null,"properties":{"requestId":"null","clientRequestId":"null","correlationRequestId":"null"}}</json>
2021-07-14T01:14:15.2558809Z | MindaroCli | ERROR | Logging handled exception: Microsoft.Rest.HttpOperationException: {"Request":{"Method":{"Method":"GET"},"RequestUri":"https://REDACTED/api/v1/name
spaces/fin-acceptance/pods","Properties":{},"Content":null,"Headers":{"Authorization":["REDACTED"]}},"Response":{"StatusCode":502,"ReasonPhrase":"Bad Gateway","Content":"<html><head><title>502 Bad Gateway</title></head>\r\n<body
><h1>DNS error</h1>\r\n<p>DNS error (the host name of the page you are looking for does not exist)<br><br>Please check that the host name has been spelled correctly.<br></p>\r\n<!--Zscaler--></body></html>\r\n","Headers":{"Server":[
"Zscaler"],"Connection":["close"],"Content-Type":["text/html"]}},"Body":null,"StackTrace":"   at k8s.Kubernetes.ListNamespacedPodWithHttpMessagesAsync(String namespaceParameter, Nullable`1 allowWatchBookmarks, String continueParamet
er, String fieldSelector, String labelSelector, Nullable`1 limit, String resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, String pretty, Dictionary`2 customHeaders, CancellationToken cancellatio
nToken)\n   at k8s.KubernetesExtensions.ListNamespacedPodAsync(IKubernetes operations, String namespaceParameter, Nullable`1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable`1 limit, St
ring resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, String pretty, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.<>c__DisplayClass
29_0.<<ListPodsInNamespaceAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.<>c__DisplayClass75_0`1.<<ClientInvok
eWrapperAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Common.Utilities.WebUtilities.RetryUntilTimeAsync(Func`2 action, TimeSpan maxWaitTime, Can
cellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.ClientInvokeWrapperAsync[T](Func`1 handler, String operation, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKuberne
tes.Common.Kubernetes.KubernetesClient.ListPodsInNamespaceAsync(String namespaceName, IEnumerable`1 labels, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Library.Client.ManagementClients.KubernetesManagementCl
ient.<>c__DisplayClass6_0.<<RefreshCredentialsAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Library.ManagementClients.ManagementClientExceptionS
trategy.<>c__DisplayClass3_0.<<RunWithHandlingAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Library.ManagementClients.ManagementClientExceptionS
trategy.RunWithHandlingAsync[T](Func`1 func, FailureConfig failureConfig)\n   at Microsoft.BridgeToKubernetes.Library.Client.ManagementClients.KubernetesManagementClient.RefreshCredentialsAsync(IProgress`1 progress, String targetNamespa
ce, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Exe.Commands.Connect.RefreshCredentialsCommand.ExecuteAsync()","Message":"Operation returned an invalid status code 'BadGateway'","Data":{},"InnerException":nu
ll,"HelpLink":null,"Source":"KubernetesClient","HResult":-2146233088}
2021-07-14T01:14:15.2583781Z | MindaroCli | ERROR | Operation returned an invalid status code 'BadGateway'\n
2021-07-14T01:14:15.6499073Z | MindaroCli | TRACE | Event: Command.End <json>{"properties":{"arguments":"refresh-credentials --namespace fin-acceptance","result":"Failed","failureReason":"Operation returned an invalid status code 'BadGa
teway'"},"metrics":{"duration":1848.0}}</json>

$ cat bridge-library-2021-07-14-01-14-14-1344.txt
2021-07-14T01:14:14.0887569Z | Library | TRACE | Pulling kubeconfig...\nOperation context: <json>{"clientRequestId":null,"correlationId":"b0464677-5cce-4795-9b1f-912f86f4ccc81626224689906:4559d60919c2:4261ee7f1ef5","requestId":null,"use
rSubscriptionId":null,"startTime":"2021-07-14T01:14:14.0054374Z","userAgent":"VSCode/1.0.120210713","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210712.3","requestHeaders":{},"loggingProperties":{"applicationName":"Librar
y","deviceOperatingSystem":"Linux 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021","framework":".NET Core 3.1.9","macAddressHash":"3424b72ee104e99be183dc60a7b751a058f3ff95747d04922fb59889f7e032b1","processId":1344,"targetEnvironmen
t":"Staging"}}</json>
2021-07-14T01:14:14.8171642Z | Library | TRACE | Event: CloudProvider <json>{"properties":{"clusterFQDNDomain":"co.nz"},"metrics":null}</json>
\n'!--Zscaler--></body></html>he page you are looking for does not exist)<br><br>Please check that the host name has been spelled correctly.<br></p>ay', Content='<html><head><title>502 Bad Gateway</title></head>
2021-07-14T01:14:15.0643992Z | Library | TRACE | Event: KubernetesManagementClient-RefreshCredentialsAsync <json>{"properties":{"result":"Failed"},"metrics":{"durationInMs":203.0}}</json>\nOperation context: <json>{"clientRequestId":"80
0b8158-9083-4d13-8d7b-314142509296","correlationId":"b0464677-5cce-4795-9b1f-912f86f4ccc81626224689906:4559d60919c2:972988002d1e","requestId":null,"userSubscriptionId":null,"startTime":"2021-07-14T01:14:14.0054374Z","userAgent":"VSCode/
1.0.120210713","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210712.3","requestHeaders":{},"loggingProperties":{"applicationName":"Library","deviceOperatingSystem":"Linux 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 202
1","framework":".NET Core 3.1.9","macAddressHash":"3424b72ee104e99be183dc60a7b751a058f3ff95747d04922fb59889f7e032b1","processId":1344,"targetEnvironment":"Staging"}}</json>
2021-07-14T01:14:15.0716929Z | Library | ERROR | Failed to Refresh Credentials
2021-07-14T01:14:15.1800658Z | Library | ERROR | Logging handled exception: Microsoft.Rest.HttpOperationException: {"Request":{"Method":{"Method":"GET"},"RequestUri":"https://REDACTED/api/v1/namespa
ces/fin-acceptance/pods","Properties":{},"Content":null,"Headers":{"Authorization":["REDACTED"]}},"Response":{"StatusCode":502,"ReasonPhrase":"Bad Gateway","Content":"<html><head><title>502 Bad Gateway</title></head>\r\n<body><h
1>DNS error</h1>\r\n<p>DNS error (the host name of the page you are looking for does not exist)<br><br>Please check that the host name has been spelled correctly.<br></p>\r\n<!--Zscaler--></body></html>\r\n","Headers":{"Server":["Zs
caler"],"Connection":["close"],"Content-Type":["text/html"]}},"Body":null,"StackTrace":"   at k8s.Kubernetes.ListNamespacedPodWithHttpMessagesAsync(String namespaceParameter, Nullable`1 allowWatchBookmarks, String continueParameter,
 String fieldSelector, String labelSelector, Nullable`1 limit, String resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, String pretty, Dictionary`2 customHeaders, CancellationToken cancellationTo
ken)\n   at k8s.KubernetesExtensions.ListNamespacedPodAsync(IKubernetes operations, String namespaceParameter, Nullable`1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable`1 limit, Strin
g resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, String pretty, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.<>c__DisplayClass29_
0.<<ListPodsInNamespaceAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.<>c__DisplayClass75_0`1.<<ClientInvokeWr
apperAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Common.Utilities.WebUtilities.RetryUntilTimeAsync(Func`2 action, TimeSpan maxWaitTime, Cancel
lationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Common.Kubernetes.KubernetesClient.ClientInvokeWrapperAsync[T](Func`1 handler, String operation, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes
.Common.Kubernetes.KubernetesClient.ListPodsInNamespaceAsync(String namespaceName, IEnumerable`1 labels, CancellationToken cancellationToken)\n   at Microsoft.BridgeToKubernetes.Library.Client.ManagementClients.KubernetesManagementClien
t.<>c__DisplayClass6_0.<<RefreshCredentialsAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Library.ManagementClients.ManagementClientExceptionStra
tegy.<>c__DisplayClass3_0.<<RunWithHandlingAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.BridgeToKubernetes.Library.ManagementClients.ManagementClientExceptionStra
tegy.RunWithHandlingAsync[T](Func`1 func, FailureConfig failureConfig)","Message":"Operation returned an invalid status code 'BadGateway'","Data":{},"InnerException":null,"HelpLink":null,"Source":"KubernetesClient","HResult":-2146233088
}
daniv-msft commented 3 years ago

Thanks @hirannair! I logged this issue to ask for their help understanding what happens: https://github.com/kubernetes-client/csharp/issues/662

SeWieland commented 2 years ago

I can confirm that I do have the exact same problem. The problem seems indeed to be related to an enforced reverse proxy (ZScaler) here as well.