Support VS 2019

[twenzel]

Please add support for Visual Studio 2019.

[p10tyr]

It should work to be honest. Did you try?

-EDIT You can install it but it wont work. Sorry. when I get some free time I will try and get to it.

[p10tyr]

unless the installer is locked to vs2017.. otherwise its just a config thing in the installer. ill check

[twenzel]

That's the problem, the installer is locked to VS2017 and the Extension gallery within VS2019 does not list your extension.

[p10tyr]

I installed VS2019 and tested it. There was an issue on installer with dependancy Please use 1.16.4 as there was some dependency that I did not need and is not installed in VS2019 anyway.

[bobted]

I have 1.16.4 installed in 2019 Pro, but get the below error when running MSTest unit tests.

System.TypeAccessException: Attempt by method 'PrestoCoverage.PrestoCoverageCore.OnTestExecutionStarting(System.Object, Microsoft.VisualStudio.TestWindow.Extensibility.OperationStateChangedEventArgs)' to access type 'Microsoft.VisualStudio.TestWindow.Controller.TestRunConfiguration' failed.
   at PrestoCoverage.PrestoCoverageCore.OnTestExecutionStarting(Object sender, OperationStateChangedEventArgs stateArgs) in C:\Projects\PrestoCoverage\PrestoCoverage\PrestoCoverage\PrestoCoverageCore.cs:line 66
   at PrestoCoverage.PrestoCoverageContainerDiscoverer.OperationState_StateChanged(Object sender, OperationStateChangedEventArgs e) in C:\Projects\PrestoCoverage\PrestoCoverage\PrestoCoverage\PrestoCoverageContainerDiscoverer.cs:line 73
   at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.OperationData.OnStateChanged(OperationStateChangedEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.OperationData.Microsoft.VisualStudio.TestWindow.Controller.IOperationData.OnRequestStateChanged(Object sender, RequestStateChangedEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.Request.SetState(RequestStates newState

Hopefully this helps.

[p10tyr]

I am looking into this. I noticed this after installing but didnt get round to it yet.

[replaysMike]

I too get this exact issue. Using 1.16.4 on VS 2019 16.0.3. Unfortunate as I was hoping to use this.

[replaysMike]

I should note for others as well, you actually need to uninstall the extension to be able to run tests again as disabling it didn't affect the error.

[p10tyr]

Sorry guys... Something has changed in VS2019 and I am not sure when I will have time to sort it out.

[p10tyr]

Well I found the problem.. They changed the TestWindows dll in VS2019 so when you try and run a test I try to use reflection to get some information about what is going on with the tests.

Why they did is beyond me but I am too stupid to work out how to get this information using the "Extensions way" because there is no docs anywhere about this. I got this working in 2017 thanks to hours of debugging Visual Studio in Visual Studio and trying all sorts of magic out.

The thing I was using in VS2017, the DLL I had to manually reference had a public sealed class and the 2019 version is internal sealed which prevents me from casting to this type. I can probably go deeper into reflection and get this... but for the love of god.. why?!

#region Assembly Microsoft.VisualStudio.TestWindow.Core, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Microsoft.VisualStudio.TestWindow.Core.dll
#endregion

namespace Microsoft.VisualStudio.TestWindow.Controller
{
    **public** sealed class TestRunConfiguration : RequestConfiguration, ITestRunRequestConfiguration, TestPlatform.ObjectModel.Client.ITestRunConfiguration, Controller.ITestRunConfigurationProvider
    {

#region Assembly Microsoft.VisualStudio.TestWindow.Core, Version=16.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Microsoft.VisualStudio.TestWindow.Core.dll
#endregion

namespace Microsoft.VisualStudio.TestWindow.Controller
{
    **internal** sealed class TestRunConfiguration :

So this is why we are getting the error message ... access type 'Microsoft.VisualStudio.TestWindow.Controller.TestRunConfiguration' failed.

[p10tyr]

Well digging in a bit deeper everything got changed to internal.

If I was conspiracy theorists I would say that one of the Microsoft guys from the Coverlet git saw this and the only reason to close this up to keep this kind of feature for Enterprise users only. But I have no proof of that, I am tired and pissed about this and just speculating. So sorry guys....

No VS2019 support!

[replaysMike]

Ah man don’t you hate when they do stuff like this? I wonder if it was to prevent not having Enterprise to get these stats. I wonder how they do it in DotCover?

[p10tyr]

Well if I play the conspiracy card I have to be fair and play the good guy theory card too.

I believe VS2019 is going to force all extensions to run async and they are probably locking down sync access like this down. The thing is the plugin and coverlet run under the VS Thread, luckily its not a massive issue because it doesn't slow down usage while writing code.. and when we run tests we usually go make a brew so nobody notices a minor speed slowdown.

So really.. I have to redo everything to be async and it would be nice if dependency injection worked from VS into extensions (which it doesnt) And all the shit I have seen is still COM hooks, weird export attributes and then my attempt with meta programming.

I have actually stared at the DotCover (and others) source code for hours. I can hardly make head and tails of it but I have literally nicked their VisualStudioExtensions file to help with a few things.

But I just cannot work out how they hook into the Test Window. Usually when I struggle with these things its because I am doing it wrong. But I cannot find any good useful information about Extension developing (like not the shitty examples that avoid using anything useful from VS) 😢

So why do I need the test window? A private object "Configuration" contains the following information about the test runner.

1. Is the debugger attached?
   1. If yes then do not attach Coverlet because coverlet modifes part of the binary and causes debugging to fail
2. Which Test projects are being run?
   1. and iterate over them to give me the exact path of the DLL so I can tell Coverlet what to use

I have spent lots of time trying to get that information from the IDE Interfaces but its just not available. The only reason I found the Configuration object was because I installed Rosyln Analysers that expose private and polymorphic things while debugging... and I spent a lot of time in that shitty little inspector window clicking into layers of properties trying to find something useful. And now they took it away.

[p10tyr]

I wonder how they do it in DotCover?

I replied before I was staring at the DotCover source code... there is no source code.. I got confused with another open source project AxoCover I think.

The way DotCover does it is by running its own TestRunner out of process. They gather the data then from their own internal gubbins.

[dozer75]

Sad thing it didn't work in 2019 since it looks to be a simple and handy extension, but could you remove 2019 from supported versions on the marketplace until this is solved? So people doesn't get confused?

[guillaume86]

Do you need TestWindow.dll for the basic watch coverage.json files and color gutters functionality? I don't mind running dotnet watch with the appropriate settings myself.

[p10tyr]

Ahh yes. I completely forgot about that aspect I am so frustrated with this automatic thing. Will try and sort that asap to get it working with 2019 in a limited state at least, until I find some other way to fix the testwindow thing

[iainnicol]

@ppumkin, you say

So why do I need the test window? A private object "Configuration" contains the following information about the test runner.

Given the trouble we've had with reflection, I have been thinking a bit about possible alternative approaches.

1. Is the debugger attached? i. If yes then do not attach Coverlet because coverlet modifes part of the binary and causes debugging to fail

I eventually realised the question to ask is: How does coverlet itself handle this? Coverlet has a package for direct VSTest DLL integration. Another package is offered for MSBuild integration, but that doesn't help us because the VS test runner uses... VSTest directly.

Now, the coverlet VSTest integration is essentially a DataCollector. The coverlet docs suggest running this collector from the dotnet test command line. But collectors can also be used from .runsettings files, including from Visual Studio.

So IMHO we don't really need any code here. Instead we just need documentation, mainly an example .runsettings file.

2. Which Test projects are being run?

And hopefully the answer here is similar: we need a DataCollector, though this time we write our own, to simply add to our list of recently run test projects. This might require some IPC between the data collector and the main PrestoCoverage instance.

What do you think? Is this worth me or you prototyping?

[p10tyr]

I know what I am trying to do is probably incorrect. I have learnt about this years ago in development. If it seems too difficult it’s probably because it’s the wrong way 😂

I am a newb to extensions and what you have proposed is a different way. Probably the correct way. So yes please try and make a branch if you will.

All we need to know is to attach coverlet before the test runs... ? And when they finish to aggregate the results.

[p10tyr]

https://github.com/Microsoft/vstest/tree/master/test/TestAssets/OutOfProcDataCollector

This looks promising. I am sure I have come across before... not sure why I decided to go down reflection route. Probably couldn’t get enough information out of that data collector.

[iainnicol]

Ah. Well hopefully I have better luck then!

I am amused by your comments about difficult things being the wrong way. Yes. But sometimes wrong code is the only way!

[p10tyr]

Please try it. I hope you can find a solution.

[iainnicol]

Good news. This approach works. The code needs tidied up, but I hope to contribute it next week.

Turns out this is bleeding edge stuff. The coverlet DataCollector was only released a week ago, the day I started looking at this! It just wasn't available when you took a look at data collectors.

[p10tyr]

Ahhh right. Very interesting. I have a feeling your findings will make it a lot easier understand now. But is this stuff is compatible with VS2017 ? Or that can still stay with the fugly reflection garbage?

Cannot wait to see how you fixed it 👍

[LoriBru]

Ehi guys! Any update on the VS2019 compatibility?

[Hviid]

@iainnicol Any news on this issue?

[iainnicol]

Ah, sorry, mainly I got distracted by other things. I will work on this this weekend.

I had a minor but required pull request rejected for Presto [edit: coverlet] itself, which is unfortunate. I may need a hack. Worst case, I suppose, isn't that bad because Presto [edit: coverlet] has already been forked.

[p10tyr]

You mean a PR for coverlet right ? Did they for coverlet too?

---

From: Iain Nicol notifications@github.com Sent: Wednesday, August 14, 2019 8:35:45 PM To: ppumkin/PrestoCoverage PrestoCoverage@noreply.github.com Cc: Piotr Kula piotr@kula.solutions; Mention mention@noreply.github.com Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

Ah, sorry, mainly I got distracted by other things. I will work on this this weekend.

I had a minor but required pull request rejected for Presto itself, which is unfortunate. I may need a hack. Worst case, I suppose, isn't that bad because Presto has already been forked.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE3T2FHQPAP6QZN7SX3QERGADA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4JWURA#issuecomment-521366084, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJRDE667A3OFNVH2MWJKJLQERGADANCNFSM4G6VA5TA.

[iainnicol]

Oops. I indeed meant coverlet.

We need coverlet to target netstandard (tonerdo/coverlet#470) so that the coverlet VS Test collector works within Visual Studio. Otherwise their VS Test collector only works from the console.

So what to to? Hopefully VS has changed and now works with netcoreapp. If not, maybe we can still consume the coverlet nuget package, but hack the metadata to look like netstandard. Else we have to fork. I know you've had to fork coverlet before, but I don't want to add a second reason for the fork!

[p10tyr]

What if we fix what we need in our fork. There is nothing changed there really so we could upgrade the pull, upgrade it to net standard ? And PR it back maybe ?

---

From: Iain Nicol notifications@github.com Sent: Saturday, August 17, 2019 2:50:30 PM To: ppumkin/PrestoCoverage PrestoCoverage@noreply.github.com Cc: Piotr Kula piotr@kula.solutions; Mention mention@noreply.github.com Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

Oops. I indeed meant coverlet.

We need coverlet to target netstandard (tonerdo/coverlet#470https://github.com/tonerdo/coverlet/pull/470) so that the coverlet VS Test collector works within Visual Studio. Otherwise their VS Test collector only works from the console.

So what to to? Hopefully VS has changed and now works with netcoreapp. If not, maybe we can still consume the coverlet nuget package, but hack the metadata to look like netstandard. Else we have to fork. I know you've had to fork coverlet before, but I don't want to add a second reason for the fork!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE4UB7TKIP47OZHTU63QE7XZNA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4QKQNA#issuecomment-522233908, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJRDE2VCBZB6A45CW2ONYTQE7XZNANCNFSM4G6VA5TA.

[cmenzi]

Any updates on this?

[p10tyr]

Not yet sorry. Will have a look next week I hope. Somebody had an idea but not sure if it worked. There are no guides to any of this 🧐😕

---

From: Cédric Menzi notifications@github.com Sent: Tuesday, August 27, 2019 11:37:55 AM To: ppumkin/PrestoCoverage PrestoCoverage@noreply.github.com Cc: Piotr Kula piotr@kula.solutions; Mention mention@noreply.github.com Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

Any updates on this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE6LDND6BPBLQL3S333QGTYXHA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5HEQNY#issuecomment-525223991, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJRDEY5YRVT4RAY7CW5WR3QGTYXHANCNFSM4G6VA5TA.

[p10tyr]

Good news. This approach works. The code needs tidied up, but I hope to contribute it next week.

Turns out this is bleeding edge stuff. The coverlet DataCollector was only released a week ago, the day I started looking at this! It just wasn't available when you took a look at data collectors.

@iainnicol I have a fork running here https://github.com/ppumkin/coverlet we can change whatever we need to get this going. I am also building the Experimental Coverlet package using that fork, which on on the VS Store.

Problem is that coverlet do not want to "open" the coverlet.core because they afraid to change API's that may cause backlash from the community... but also they not offering any solutions so i just forked it and told them im going to build the core one on my own responsibility. They said OK.

So if you got a coverlet PR, do it against my fork, i will rebuild the package that this project uses and lets get VS2019 working... ??! 😄

[p10tyr]

I released 1.16.5 that will not cause that dumb ass exception to show any more..

but the automagic coverage DOES NOT WORK IN 2019 YET

You can only use the watch folder function in VS2019 for NOW.. atleast something for the time being.

Sorry for taking so long to get to it.

[jotatoledo]

Just found out your extension after almost giving up hope on coverage-tools for VS.

Sorry for taking so long to get to it.

Thanks for the amazing work. This is OSS, so dont feel sorry for progress taking its time!

[adrianhara]

I'm using VS2019 Professional and get no gutters. The json report is generated under c:\coverlet\xunittestcoverage.json, looks good, the extension is installed, but no gutters. Any idea what I could try?

[brianhill-bfs]

I have the same issue as @adrianhara with VS2019 Professional. I have coverage output, but nothing loaded in VS.

[siuarima9]

I'm using vs2019 but this extension isn't working..

[eduardocp]

@ppumkin

I think this resolve the problem of internal classes/methods:

dynamic testRequestConfiguration = stateArgs.Operation.GetType()
    .GetProperty("Configuration")
    .GetValue(stateArgs.Operation);

var sources = testRequestConfiguration.GetType()
    .GetProperty("TestSources")
    .GetValue(testRequestConfiguration);

//if (testRequestConfiguration.Debug)
//  return;

foreach (string testDll in sources)
{
    ...
}

I also saw that the Debug property has been removed...

[dozer75]

Tried 1.16.6 today with Visual Studio 16.3.10 and still not working. Getting:

[2019-11-28 9:31:24.778 pm Error] System.MissingMethodException: Method not found: 'Boolean Microsoft.VisualStudio.TestWindow.Controller.TestRunConfiguration.get_Debug()'.
   at PrestoCoverage.PrestoCoverageCore.OnTestExecutionStarting(Object sender, OperationStateChangedEventArgs stateArgs)
   at PrestoCoverage.PrestoCoverageContainerDiscoverer.OperationState_StateChanged(Object sender, OperationStateChangedEventArgs e) in C:\Projects\PrestoCoverage\PrestoCoverage\PrestoCoverage\PrestoCoverageContainerDiscoverer.cs:line 73
   at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.OperationData.<>c__DisplayClass44_0.<OnStateChanged>b__0()
   at Microsoft.VisualStudio.TestWindow.Extensibility.ILoggerExtensions.CallInternalWithLogging(ILogger2 log, Action action, Boolean shouldThrow)
--- End of stack trace from previous location where exception was thrown ---

[pcbl]

same issue as @dozer75 , when running the tests on Vs 2019 16.3.10.

[eduardocp]

@ppumkin had already commented above:

I released 1.16.5 that will not cause that dumb ass exception to show any more..

but the automagic coverage DOES NOT WORK IN 2019 YET

You can only use the watch folder function in VS2019 for NOW.. atleast something for the time being.

Sorry for taking so long to get to it.

[dozer75]

@eduardocp Well, he stated that the "dumb ass exception" shouldn't occur anymore after 1.6.15, but it does in 1.6.16, that's why I added the message... :)

To clarify to anyone else what's this means: If you run tests in Visual Studio 2019 it will not do auto coverage when you run the test using test explorer in visual studio (you'll get that "dumb ass exception")...

However; if you run manually using dotnet test /p:CollectCoverage=true /p:CoverletOutput=<Path> where <Path> is the folder configured in Tools -> Options -> PrestoCoverage -> General Settings -> Path the coverage will be displayed.

[AXMIM]

I'm unable to install it on VS Professionnal 2019 Version 16.2.5 I get the following message. I rebooted a few times already to no avail. It is sad to see this extension get screwed up by VS 2019.

2019-12-20 11:41:35 - Microsoft VSIX Installer 2019-12-20 11:41:35 - ------------------------------------------- 2019-12-20 11:41:35 - vsixinstaller.exe version: 2019-12-20 11:41:35 - 16.4.1057 2019-12-20 11:41:35 - ------------------------------------------- 2019-12-20 11:41:35 - Command line parameters: 2019-12-20 11:41:35 - C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service\vsixinstaller.exe,/appidinstallpath:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\devenv.exe,/skuName:Pro,/skuVersion:16.2.29306.81,/appidname:Microsoft Visual Studio Professional 2019,/culture:en-US,/noep,C:\Users\maxiveil\AppData\Local\Temp\VSIXjfohq50j.vsix 2019-12-20 11:41:35 - ------------------------------------------- 2019-12-20 11:41:35 - Microsoft VSIX Installer 2019-12-20 11:41:35 - ------------------------------------------- 2019-12-20 11:41:36 - Setup instance 8d590194 requires a reboot. Reboot the machine and try again. 2019-12-20 11:41:36 - System.InvalidOperationException: Setup instance 8d590194 requires a reboot. Reboot the machine and try again. à VSIXInstaller.SetupExtensions.GetLaunchableInstances(IQuery query) à VSIXInstaller.SupportedSKUs.EnumerateIsolatedInstalls(Action1 callback) à VSIXInstaller.SupportedSKUs.AddInstalledLocationBasedSKUs(IntPtr userToken) à VSIXInstaller.SupportedSKUs.InitializeSupportedSKUs(IntPtr userToken) à VSIXInstaller.ExtensionService.InitializeSupportedSKUs(ICommandLineData cmdLineData, IntPtr duplicatedUserToken) à VSIXInstaller.App.Initialize(Boolean isRepairSupported) à VSIXInstaller.App.Initialize() à System.Threading.Tasks.Task1.InnerInvoke() à System.Threading.Tasks.Task.Execute() --- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée --- à Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)

[BDomzalski]

It's nearly half-year since @iainnicol checked that the approach with DataCollectors works and a fork of Coverlet has been created. Any progress? I know from own experience that finding time might be almost impossible. I would like to see this working so maybe I could help?

[p10tyr]

Hi Bartosz. Yea I have been meaning to get back to this. I would probably spend some time tonight to look at the forks ? I didn’t see where the data collectors have been used to work ? If you can put some links in it will Make it faster for me to discover this and fix it maybe

---

From: Bartosz Domżalski notifications@github.com Sent: Monday, February 10, 2020 10:48:05 AM To: ppumkin/PrestoCoverage PrestoCoverage@noreply.github.com Cc: Piotr Kula piotr@kula.solutions; Mention mention@noreply.github.com Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

It's nearly half-year since @iainnicolhttps://github.com/iainnicol checked that the approach with DataCollectors works and a fork of Coverlet has been created. Any progress? I know from own experience that finding time might be almost impossible. I would like to see this working so maybe I could help?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE7NSMYB6I5BGYT7NX3RCEWGLA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELIBSMI#issuecomment-584063281, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAJRDE5CP3SAREZJ4KBBWZ3RCEWGLANCNFSM4G6VA5TA.

[BDomzalski]

I did a quick look around the forks but did not spot where it could be that those collectors were used. I only infer that base on the following two comments: description of possible solution and approach works.

[p10tyr]

Ok. I think they thought that would work. Will have to spend some time around it.

---

From: Bartosz Domżalski notifications@github.com Sent: Monday, February 10, 2020 11:21:43 AM To: ppumkin/PrestoCoverage PrestoCoverage@noreply.github.com Cc: Piotr Kula piotr@kula.solutions; Mention mention@noreply.github.com Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

I did a quick look around the forks but did not spot where it could be that those collectors were used. I only infer that base on the following two comments: description of possible solutionhttps://github.com/ppumkin/PrestoCoverage/issues/17#issuecomment-499590687 and approach workshttps://github.com/ppumkin/PrestoCoverage/issues/17#issuecomment-502117782.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE64F2BLVYJF6PTIO5LRCE2EPA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELIEXVA#issuecomment-584076244, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAJRDEZEZYAXP772BKZ2LZLRCE2EPANCNFSM4G6VA5TA.