search

microsoft / ConcordExtensibilitySamples

Visual Studio Debug Engine Extensibility Samples
Other
121 stars 50 forks source link

Calling back into the default expression evaluator doesn't work with arrays #75

Open akbyrd opened 2 years ago

akbyrd commented 2 years ago

This may be an issue with my implementation, but I've noticed that when I try to create a visualizer that just kicks the work back to the default expression evaluator I get different results than when I have no visualizer at all. In particular, I start to see ??? in the display string for array elements in the Watch window.

In this screenshot the NoVis type has no custom visualizer at all. Notice that it shows the value of the single array element in the display string. The DefaultVis type has a custom visualizer but all it does is callback into EvaluateExpressionCallback from EvaluateVisualizedExpression and set pUseDefaultEvaluationBehavior to true in UseDefaultEvaluationBehavior.

I'm using VS2022 (17.0.4) here. image

In VS2019 (16.11.8) I get slightly different results. I can run the same solution (after downgrading the Microsoft.VSSDK.BuildTools package and build tools version) and it works correctly. However, it will exhibit the same issue if I set pUseDefaultEvaluationBehavior to false and then use the callbacks to do default visualization in the rest of the IDkmCustomVisualizer functions. image

vs-native-debug-visualizer-github.zip

akbyrd commented 2 years ago

After poking at this some more, I think VS2019 shows the same behavior. I'm not sure why it appeared to be working previously. Probably something with the extension not being updated/replaced properly.

When I have an array variable the root visualized expression name is L"[0]" and the full name is L"()[0],nd". I use the full name to create a language expression and hand that to EvaluateExpressionCallback. Obviously, it doesn't know what do with that expression.

It's only the display string on the root that is messed up. When I expand it the root visualized expression full name is L"avDefault[0]" as expected. Though it does mean that the name shows up funny in the Watch window. image

I see a similar issue if I have a pointer value and I use ,1 to tell the Watch window it's an array. Name and full name are both L"" when expanding the root object. image

Note that the VSIX project may not load. I added the Microsoft.VSSDK.BuildTools nuget package and added Microsoft.VsSDK.targets as a build customization. When redistributing the project (without the nuget package directory) the project won't load because the targets file is missing. The targets file can't be downloaded unless you can load the project and let nuget re-download packages. Fun chicken and egg problem.