opencover hangs after running most unit tests (MSTests, through Gallio, launched by Sonar)

[arkrohne]

The build server is running Windows Server 2008, and I'm using openCover 3.0.121.

openCover is being run from Sonar, though I also tried running it directly using the same command. After running through most of the unit tests, it slows to a crawl, with the last unit test giving an OutOfMemoryException, and hangs at that point.

The full command that Sonar generates is as follows:

C:\Users\Administrator\AppData\Local\Apps\OpenCover\OpenCover.Console.exe -register -target:"C:\Program Files\Gallio\bin\Gallio.Echo.exe" -targetdir:C:\Jenkins\workspace\TVACodeLibrary\Main\Source\.sonar "-targetargs:\"/r:IsolatedAppDomain\" \"/report-directory:C:\Jenkins\workspace\TVACodeLibrary\Main\Source\.sonar\" \"/report-name-format:TVA.Core.Tests.dll.gallio-report\" \"/report-type:Xml\" \"/f:*\" \"C:\Jenkins\workspace\TVACodeLibrary\Main\Source\Libraries\Tests\TVA.Core.Tests\..\..\..\..\Build\Output\Debug\Libraries\Tests\TVA.Core.Tests.dll\"" "-filter:+[TVA.Core]* +[TVA.Communication]* +[TVA.Security]* +[TVA.Web]* +[TVA.Windows]* +[ConfigCrypter]* +[UDPRebroadcaster]* +[UDPRebroadcasterConsole]* +[ConfigEditor]* +[TVA.ServiceProcess]* +[TVA.ServiceModel]* +[TVA.ServiceBus]* +[TCPRebroadcaster]* +[TCPRebroadcasterConsole]* +[DataMigrationUtility]* " -mergebyhash -output:C:\Jenkins\workspace\TVACodeLibrary\Main\Source\.sonar\TVA.Core.Tests.dll.coverage-report.xml -log:Debug

The project is an open-source library we're using: http://tvacodelibrary.codeplex.com

[sawilde]

Okay I'll need your help as I do not have your setup

I downloaded Gallio and TVA... but I couldn't get any tests to run. Gallio went through the motions but at the end came up with 0 of 0 tests....

Once I am sure I have Gallio working properly I'll then run it under OpenCover.

NOTE: Do you know that Gallio has PartCover integrated (the precursor) to OpenCover?

[arkrohne]

If it'd help, I'll upload the contents of the directory from my setup.

I was not aware that Gallio had PartCover. I learned of opencover/partcover when I was setting up Sonar, and the C# setup instructions from Sonar recommended I install opencover. When I encountered this issue, I tried partcover instead and while it ran fine, it claimed that I had 0% coverage (despite Gallio running all 800 or so tests).

[sawilde]

OpenCover is newer and also has branch coverage and supports 64bit (unlike PartCover).

The 64bit may be your issue when working with PartCover.

I've got everything to compile just can't get Gallio to run the tests.

[arkrohne]

The server I'm running on is 32bit. It sounds like OpenCover is what I want to use, base on your description.

Evidently, the only tests cases are in the TVA.Core.Tests.dll. When I build it with Jenkins as a debug build, that file is in C:\Jenkins\workspace\TVACodeLibrary\Main\Build\Output\Debug\Libraries\Tests\TVA.Core.Tests.dll, and they work run both from the command line and from the Icarus GUI

I'm not sure why those tests don't run on your system.

[sawilde]

What sort of command line do you use i.e. without all the OpenCover wrapping?

[arkrohne]

Just to verify that Gallio is running the tests, I ran "C:\Program Files\Gallio\bin\Gallio.Echo.exe" TVA.Core.Tests.dll and it showed all the tests running.

[arkrohne]

If it's helpful, here's a link to the tests, as I have them compiled:

[sawilde]

it seems if I use /r:IsolatedAppDomain as shown then I get

Stop time: 11:32 PM (Total execution time: 11.652 seconds)

0 run, 0 passed, 0 failed, 0 inconclusive, 0 skipped

but if I drop that argument then I get

Stop time: 11:33 PM (Total execution time: 51.476 seconds)

871 run, 607 passed, 236 failed, 28 inconclusive, 0 skipped

At least I can run the tests (the number of failures look bad though but I'll ignore for now - NMP)

[arkrohne]

Well, the number of failures matches what I get, so that's good.

[sawilde]

Okay the issue is to do with the Pattern(De)Compressor class - it is generating a lot of information with visit counts > 6 million per sequence point (and that is after I stopped the test run early).

OpenCover is having trouble keeping up with the flood of information so it is just buffering up and eventually leading to the out of memory exception you reported.

I'll look into a way to sort this but it won't be quick, in the meantime you can exclude the class from coverage for now with a filter -[TVA.*]*.PatternCompressor -[TVA.*]*.PatternDecompressor

using that filter I get the following coverage

Stop time: 12:20 AM (Total execution time: 125.661 seconds)

871 run, 602 passed, 241 failed, 28 inconclusive, 0 skipped

Committing...
Visited Classes 27 of 212 (12.7358490566038)
Visited Methods 872 of 4111 (21.2113840914619)
Visited Points 2922 of 24491 (11.930913396758)
Visited Branches 252 of 6534 (3.85674931129477)

==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 27 of 256 (10.546875)
Alternative Visited Methods 872 of 4393 (19.8497609833827)

I hope this helps

[arkrohne]

This helps tremendously. Thank you; we'll run with those exclusions for now, and upgrade whenever you have a update for this.

Thanks!

[sawilde]

Raised new issue to deal with memory handling issue found