Bridge to Kubernetes does not hit any debugging symbols at all in Visual Studio 2019

[Scaramanga77]

[Scaramanga77]

I have docker desktop running and a build deployed in debug configuration. I launch the bridge and the ui starts as expected but no debugging symbols are hit.

[amsoedal]

HI @Scaramanga77, how are you running Bridge (VS/VS Code?), and how are you running your application (launch configuration in VS Code, through the command line)?

[Scaramanga77]

Hi, I am running it in Visual Studio 2019. I have a kubernetes deployment via docker desktop on Windows running in Linux container mode. The code in question is .NET 5. The app starts via the debugger but all breakpoints are telling me no symbols attached. The build is in Debug mode.

[Scaramanga77]

This is completely broken I have a simple api that I am trying to put breakpoints on a call to some middleware. No symbols are loaded and debugging does not work at all:

[amsoedal]

@Scaramanga77, what happens if you debug without Bridge? Do you see your breakpoints getting hit in that case? Also could you share how your Bridge profile is set up?

[Scaramanga77]

@amsoedal If I debug without bridge for example using docker run option in Visual Studio it is working fine. The Bridge profile is just the out of the box configuration that gets applied through Visual Studio. In my case:

Is there something else I need to configure?

My launchsettings.json bridge settings are:

[Scaramanga77]

My kubernetes setup is as follows:

I do not have any launch.json defined I just run as above straight out of the box. Am I missing a step?

[Scaramanga77]

Any update on this? It seems broken:

2021-11-24T16:59:19.4932791Z | Library | TRACE | Pulling kubeconfig...\nOperation context: {"clientRequestId":null,"correlationId":"cd143050-ae87-41d2-b0e7-496c2bc6975a:001d66952107:e6a94234703b","requestId":null,"userSubscriptionId":null,"startTime":"2021-11-24T16:59:19.4246832Z","userAgent":"LPK/2.1.20210515.1-887601f7 VisualStudio/16.11.31911.196","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210421.2","requestHeaders":{},"loggingProperties":{"applicationName":"Library","deviceOperatingSystem":"Microsoft Windows 10.0.18363","framework":".NET Core 3.1.21","macAddressHash":"9ac5bb0d7f266e80457b536ce8a400c4d7fd907d43fd34d4056520cc5030104e","processId":51416,"targetEnvironment":"Production"}} 2021-11-24T16:59:19.7132978Z | Library | TRACE | Event: CloudProvider {"properties":{"clusterFQDNDomain":"DockerInternal"},"metrics":null} 2021-11-24T16:59:20.1510077Z | Library | TRACE | Dependency: Kubernetes {"target":"ListServicesInNamespaceAsync","success":true,"duration":null,"properties":{}} 2021-11-24T16:59:20.2700747Z | Library | TRACE | Dependency: Kubernetes {"target":"ListPodsInNamespaceAsync","success":true,"duration":null,"properties":{}} 2021-11-24T16:59:20.2731422Z | Library | TRACE | Detected headless service with selectors. Using selectors to look for endpoints...\nOperation context: {"clientRequestId":null,"correlationId":"cd143050-ae87-41d2-b0e7-496c2bc6975a:001d66952107:3fcdc9e996fa","requestId":null,"userSubscriptionId":null,"startTime":"2021-11-24T16:59:19.4246832Z","userAgent":"LPK/2.1.20210515.1-887601f7 VisualStudio/16.11.31911.196","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210421.2","requestHeaders":{},"loggingProperties":{"applicationName":"Library","deviceOperatingSystem":"Microsoft Windows 10.0.18363","framework":".NET Core 3.1.21","macAddressHash":"9ac5bb0d7f266e80457b536ce8a400c4d7fd907d43fd34d4056520cc5030104e","processId":51416,"targetEnvironment":"Production","isRoutingEnabled":false}} 2021-11-24T16:59:20.2869921Z | Library | TRACE | Detected headless service with selectors. Using selectors to look for endpoints... 2021-11-24T16:59:20.2874641Z | Library | TRACE | Detected headless service with selectors. Using selectors to look for endpoints... 2021-11-24T16:59:20.3108077Z | Library | TRACE | Dependency: Kubernetes {"target":"ListEndpointsInNamespaceAsync","success":true,"duration":null,"properties":{}} 2021-11-24T16:59:20.3108067Z | Library | TRACE | Dependency: Kubernetes {"target":"ListEndpointsInNamespaceAsync","success":true,"duration":null,"properties":{}} 2021-11-24T16:59:20.3108067Z | Library | TRACE | Dependency: Kubernetes {"target":"ListEndpointsInNamespaceAsync","success":true,"duration":null,"properties":{}} 2021-11-24T16:59:20.3158540Z | Library | TRACE | Found 3 endpoint(s) matching headless service 'redis-internal-headless' 2021-11-24T16:59:20.3158540Z | Library | TRACE | Found 2 endpoint(s) matching headless service 'kafka-internal-headless' 2021-11-24T16:59:20.3158540Z | Library | TRACE | Found 2 endpoint(s) matching headless service 'kafka-internal-zookeeper-headless' 2021-11-24T16:59:20.3245356Z | Library | TRACE | Event: KubernetesRemoteEnvironmentManager-GetReachableServices {"properties":{"result":"Failed"},"metrics":{"durationInMs":544.0}} 2021-11-24T16:59:20.3280892Z | Library | TRACE | Event: ConnectManagementClient-GetElevationRequests {"properties":{"result":"Failed"},"metrics":{"durationInMs":569.0}} 2021-11-24T16:59:20.3288685Z | Library | ERROR | Failed to get elevation requests. 2021-11-24T16:59:20.3512720Z | Library | ERROR | Logging handled exception: System.NullReferenceException: {"ClassName":"System.NullReferenceException","Message":"Object reference not set to an instance of an object.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.Collections.Generic.Dictionary2.FindEntry(TKey key)\r\n at System.Collections.Generic.Dictionary2.ContainsKey(TKey key)\r\n at Microsoft.BridgeToKubernetes.Library.Connect.KubernetesRemoteEnvironmentManager.<>cDisplayClass31_0.<_CollectServicesToRouteAsync>b3(V1Endpoints e)\r\n at System.Collections.Generic.IEnumerableExtensions.ExecuteForEach[T](IEnumerable1 items, Action1 action, CancellationToken cancellationToken)\r\n at Microsoft.BridgeToKubernetes.Library.Connect.KubernetesRemoteEnvironmentManager.<>cDisplayClass31_0.<<_CollectServicesToRouteAsync>b0>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at Microsoft.BridgeToKubernetes.Library.Connect.KubernetesRemoteEnvironmentManager.GetReachableServicesAsync(String namespaceName, ILocalProcessConfig localProcessConfig, IProgress1 progress, CancellationToken cancellationToken)\r\n at Microsoft.BridgeToKubernetes.Library.ManagementClients.ConnectManagementClient.<>c__DisplayClass13_0.<<GetElevationRequestsAsync>b__0>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at Microsoft.BridgeToKubernetes.Library.ManagementClients.ManagementClientExceptionStrategy.RunWithHandlingAsync[T](Func1 func, FailureConfig failureConfig)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467261,"Source":"System.Private.CoreLib","WatsonBuckets":null}

[amsoedal]

Hi @Scaramanga77, sorry for the delayed response. It looks like you're debugging headless services -- I recommend using our VSCode extension because our VS extension is lagging behind in terms of the latest fixes. Can you give VS Code a go and see if it works for you? Similar issues here: #209, #237

[Scaramanga77]

Hi. I have tried the same with VS code but I am getting the bridge connect ok but am still not getting any debugging symbols attached. Here is my config:

{ "configurations": [ { "name": ".NET Core Launch (web) with Kubernetes", "type": "coreclr", "request": "launch", "preLaunchTask": "bridge-to-kubernetes.compound", "program": "${workspaceFolder}/src/api/bin/Debug/net5.0/Sigma.Catalog.Api.dll", "args": [], "cwd": "${workspaceFolder}/src/api", "stopAtEntry": false, "env": { "ASPNETCORE_ENVIRONMENT": "Development", "ASPNETCORE_URLS": "http://+:8080" }, "sourceFileMap": { "/app": "${workspaceFolder}/src/api" } }, { "name": ".NET Core Launch (web)", "type": "coreclr", "request": "launch", "preLaunchTask": "build", "program": "${workspaceFolder}/src/api/bin/Debug/net5.0/Sigma.Catalog.Api.dll", "args": [], "cwd": "${workspaceFolder}/src/api", "stopAtEntry": false, "serverReadyAction": { "action": "openExternally", "pattern": "\bNow listening on:\s+(https?://\S+)" }, "env": { "ASPNETCORE_ENVIRONMENT": "Development" }, "sourceFileMap": { "/app": "${workspaceFolder}/src/api" } } ] }

with the below structure:

[amsoedal]

@Scaramanga77 I tried to reproduce the issue by debugging with VS Code and targeting a pod running in K8s on Docker desktop, but all my breakpoints got hit. I'm trying to understand what might be different about your setup where this reproduces -- apologies for asking again, but when you said that you can debug with VS docker run sometimes, do you ever run the app un-containerized? What happens then? I'll also ask my colleagues in parallel to see if anyone might have insight on this.

[Scaramanga77]

Hi,

Would it be possible to arrange a chat over google meets and I can demo the exact issue?

[amsoedal]

@Scaramanga77 yes, that would be possible. Can you send a mail to BridgeToKubernetes@microsoft.com and we can coordinate?

[Scaramanga77]

Hi, I have now got partial success with the update that you sent me but only if I run via docker option in Visual Studio. If I attempt to run the project by name which spins up kestrel no breakpoint symbols are loaded. If my service calls other services I am then unable to connect to them. I get a 404 even if I add the services into the KubernetesLocalProcessConfig.yaml.

[Scaramanga77]

Is there any update to this?

[angelobreuer]

If you do not have embedded debugging symbols in the DLL (which is the default option, as far as I know), then Visual Studio has to find the symbol paths as they are not integrated with the DLL.

There are two solutions to that:

(1) either you include the debug symbols in the exported assembly (which can bloat the resulting assembly)

Select to embed debug symbols in DLL/EXE: (Transcription: Embedded in DLL/EXE, cross-platform portable)  or alternatively, add the following property to your project build configuration (or Directory.Props): 

(2) or, tell Visual Studio during debugging where your debug symbols are located.

Open the *Modules* window **during** debug mode. Click `Debugging` > `Windows` > `Modules`.  Then click on the assembly you want to load debug modules for, and click on **Load Modules**:  Visual Studio should mostly be able to find the location of the debug assembly automatically, if not a file open dialog appears where the path of the .pdb can be selected. The PDB is normally stored along with the .dll/.exe in the build directory. If the debug symbols were loaded successfully, then the following should appear:  (Relevant line: *Symbols loaded*) If you did everything correctly, breakpoints should be hit: 

Another cause could be, that you enabled Only my code (enabled by default), if Visual Studio does not classify the assembly as "your own code" then it skips it to improve the debug experience (loading tons of debug symbols could take hours) (This would be very uncommon, but could also be relevant).

If nothing of the above working, Visual Studio should at least provide an informational message why the debug symbol load failed for a module in the module window (mentioned in the second possible solution).

For example:

(Transcription: PDB file could not be found, or could not be opened.)

Hope I could provide any help.

[Scaramanga77]

I have tried this and it does not work. I have fallen back to vscode and I have the same problem:

Management.Worker.csproj

tasks.json

launch.json

I then configure bridge and no breakpoints are hit:

Note: that there is no message in the loader of attempting to even search for .pdb files.

However if I disconnect from the cluster and run the application breakpoints are hit perfectly fine:

Note: in this run the attempt to load the symbols is happening fine and my breakpoints are active.

The debugger console output shows the following:

symbols to be resolved later

symbols not loaded

You can see here for some reason it believes that the code has changed and hence no symbols. However, it has only rebuilt due to the sequencing of the task configuration created by bridge.

[joshy1990]

is there any updates in regards to this issue? i have the same issue but using visual studio 2022. i have followed the guide and its still not working.

[joshy1990]

doesn't matter where I put them it just doesn't stop the application for debugging. ive also checked JustMyCode to see if it will make a difference but it doesn't.

[IQHT-DGH]

I have having the exact same issue as @joshy1990 connecting to kubernetes in an AKS cluster has anyone managed to resolve this? If I could get this working I could move my whole team to developing with this method making life so much easier!