microsoft / VSExtensibility

A repo for upcoming changes to extensibility in Visual Studio, the new extensibility model, and language server protocol.
MIT License
351 stars 41 forks source link

Visualizer test call / VisualizerDevelopmentHost fairly hosed with json FormatterPolicy and custom VisualizerObjectSource #369

Open mitchcapper opened 1 month ago

mitchcapper commented 1 month ago

This repo is for next gen plugins I believe, but I am not sure where I would file this for current gen.

I could be wrong on this, there is no official sample visualizer I am aware of, but I have tested most recent project visualizers trying to get the test hosting to work without luck.

I am sure you have visualizer telemetry data but at least from GH code seems there are not a large amount of debug visualizers certainly fewer that use the new FormatterPolicy.Json. I tried all that seemed updated in the last 6 months and couldn't get any working out of the box.

To be clear the following is what seems to be impossible, likely I am guessing after the 17.6 changes at a minimum:

) Create a visualizer that implements VisualizerObjectSource for the debuggee side ) Add a helper for testing to the debugger side like: https://learn.microsoft.com/en-us/visualstudio/debugger/how-to-test-and-debug-a-visualizer?view=vs-2022 note you have to specify the visualizer directly, oddly the default VisualizerObjectSource somehow still passes back binary serialized data even when json is set as the formatter.
So this should be your tri arg call like: var visualizerHost = new VisualizerDevelopmentHost(obj, typeof(DynamicDebuggerVisualizer), typeof(DebuggeeObjectSource));

Create a console app net472 call your test function directly from code.

The issues I ran into: *) You need to install a helper resolver to help VS load the dlls from C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Packages\Debugger\Visualizers

*) Without copying the Microsoft.VisualStudio.DebuggerVisualizers.dll locally you get type mismatches, even when the version you reference is the one directly from the VS Visualizer folder C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Packages\Debugger\Visualizers\net4.6.2

*) Using the official nuget package doesn't end up working with or without copying (its a ref dll so makes sense on copying)

*) If you copy it locally you will get further, but the way VS tries to load its serializers (even when formatter is not set to NewtonsoftJson it seems internally it still uses the newtonsoft dll) it ends up trying an absolute path for the serializer load based on the dll location resulting in it trying to load from c:\YourProject\Bin\Debug\Microsoft.VisualStudio.DebuggerVisualizers.dll\Additional path details. Now obviously the DLL cannot be a folder and a file at the same time so this is pretty hard to work around. Load the dll from elsewhere and mismatches if the file exists in the Debug folder it bypasses the resolver anyway and always loads it.

To get it to work I had to do the above (custom resolver + copy local the same version VS loads) then avoid using the serializer/deserializer code, dont use SerializeAsJson serialize ourselves and then write to the stream directly.

Doing this can get it to work both with direct test calls, and then using the visualizer in the debugger normally (hover and view).

This also will allow direct calls in 17.6+ to work even though it seems the latest betas have potentially broken further:

https://developercommunity.visualstudio.com/t/custom-debug-visualizers-broken-in-Versi/10652170 potentially: https://github.com/microsoft/VSExtensibility/issues/361

for the normal visualizer option.

All and all many hours trying to get either to work either with test visualizers I created for any visualizers I could find on GH in the last 6 months with 17.6+ changes (or installed directly from the extensions store). Maybe it is local install related although happens on a clean /rootsuffix and an empty Visualizers user folder. In addition the dependency loading code seems pretty baked in.

mpeyrotc commented 3 weeks ago

Hello @mitchcapper, thank you for reporting this issue! You are correct, the documentation you are referring to is for the old way of writing visualizers. We did several updates to the underlying visualizer infrastructure, so it is likely that there are bugs in the test host and the docs need some updating. I will let you know once the docs are updated and if we find and fix some bugs 😄 .