microsoft / mindaro

Bridge to Kubernetes - for Visual Studio and Visual Studio Code
MIT License
307 stars 106 forks source link

50+ dsc check-credentials is launched at once #141

Open nwns opened 3 years ago

nwns commented 3 years ago

Describe the bug While using vscode is open, /home/nigel/.config/Code/User/globalStorage/mindaro.mindaro/file-downloader-downloads/bridge/dsc check-credentials --namespace default processes are spawned. It seems to scale with the number of vscode instances open (in this case I had 7).

Yesterday this bug caused my laptop to hang for over 5 minutes before I gave up and hard rebooted it.

To Reproduce This seems to be done periodically while vscode is open with this extension installed and enabled. Now that I have figured out what is causing this, I have disabled this plugin and will report back if it solves the problem.

This seems to occur more (or only) when I am using the commandline kubectl going about various sysadmin tasks within a vscode terminal or a normal LXTerm.

Expected behavior To not spawn so many processes at once (and for apparently redundant reasons since they all seem to be the same).

Logs

2021-03-26T22:26:17.5132168Z | Library | TRACE | Event: CloudProvider <json>{"properties":{"clusterFQDNDomain":"Remote IP"},"metrics":null}</json>
2021-03-26T22:28:04.0043247Z | Library | TRACE | Dependency: Kubernetes <json>{"target":"ListPodsInNamespaceAsync","success":true,"duration":null,"properties":{}}</json>
2021-03-26T22:28:04.9426665Z | Library | TRACE | Event: KubernetesManagementClient-CheckCredentialsAsync <json>{"properties":{"result":"Succeeded"},"metrics":{"durationInMs":104518.0}}</json>\nOperation context: <json>{"clientRequestId":"567e2124-41c5-4ef2-b2e8-8dbce6e600f2","correlationId":"106f453d-2996-4d90-8c66-e6763f64ab871616704839396:348abf9d40e6:a02c05045299","requestId":null,"userSubscriptionId":null,"startTime":"2021-03-26T22:26:02.7231371Z","userAgent":"VSCode/1.0.120210322","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210319.3","requestHeaders":{},"loggingProperties":{"applicationName":"Library","deviceOperatingSystem":"Linux 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30)","framework":".NET Core 3.1.6","macAddressHash":"1b244461a7e1404da677a4230e2abf9372ead3b315d769201e387a6d7c2a8524","processId":17614,"targetEnvironment":"Production"}}</json>
2021-03-26T22:27:54.0056079Z | MindaroCli | TRACE | Event: Command.Start <json>{"properties":{"arguments":"check-credentials --namespace default","isRoutingEnabled":"false"},"metrics":null}</json>\nOperation context: <json>{"clientRequestId":null,"correlationId":"65360e87-4741-4181-862b-bfdfe01dee171616788058023:a1efa63b3ae0","requestId":null,"userSubscriptionId":null,"startTime":"2021-03-26T22:26:56.4889351Z","userAgent":"VSCode/1.0.120210322","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210319.3","requestHeaders":{},"loggingProperties":{"applicationName":"MindaroCli","deviceOperatingSystem":"Linux 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30)","framework":".NET Core 3.1.6","macAddressHash":"1b244461a7e1404da677a4230e2abf9372ead3b315d769201e387a6d7c2a8524","processId":27065,"targetEnvironment":"Production","commandId":"a1efa63b3ae0"}}</json>
2021-03-26T22:28:01.0337533Z | MindaroCli | TRACE | Running Microsoft.BridgeToKubernetes.Exe.Commands.Connect.CheckCredentialsCommand...
2021-03-26T22:28:01.3529071Z | MindaroCli | TRACE | Initializing progress reporter...
2021-03-26T22:28:16.7982826Z | MindaroCli | TRACE | Event: Command.End <json>{"properties":{"arguments":"check-credentials --namespace default","result":"Succeeded"},"metrics":{"duration":28865.0}}</json>

Environment Details Client used VS Code v1.0.120210322

$ code --version
1.54.3
2b9aebd5354a3629c3aba0a5f5df49f43d6689f8
x64
Linux  4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux

Fully up to date Debian 10 (Buster)

Additional I have not actually used this vs code extension and have had this extension installed for a couple of months and it has not been an issue until this week.

rakeshvanga commented 3 years ago

@nwns Thanks for reporting this issue and also investigating the cause for the hang. We have added a new feature to support AAD enabled K8s clusters and this might be causing the issue. I would log a bug on our side and get this fixed soon.

daniv-msft commented 3 years ago

@nwns Thanks again for reporting this issue! We made good progress on the investigation. This issue is due to a bug on our side where we subscribe multiple times to changes on the current kubeconfig file. This was progressively creating more and more calls to dsc check-credentials, which resulted in the hang you've been experiencing.

We have a fix, and are presently validating that it doesn't create other issues before releasing it.

daniv-msft commented 3 years ago

@nwns We released a new version of our VS Code extension containing the fix for this one. Could you please validate that it doesn't reproduce anymore on your side? Thanks again for reporting this issue!