microsoft / codecoverage

MIT License
79 stars 11 forks source link

Skipped Module reason no_symbols #110

Closed ItaloCensi closed 6 months ago

ItaloCensi commented 6 months ago

I am attempting to utilize the Microsoft.TestPlatform NuGet package in order to avoid having to install Visual Studio on build agents, but I am unable to collect code coverage data. My target DLL is appearing in the skipped modules with the reason listed as "no_symbols".

I have verified that the PDB file has symbols generated and I do not see any issues there. When executing the same commands on a machine that has Visual Studio 2022 installed, everything works as expected and I can successfully obtain code coverage.

It seems the Microsoft.TestPlatform package alone is not sufficient for code coverage collection without Visual Studio installed, despite my target DLL having symbol information available.

I would welcome any suggestions on how to properly configure the Microsoft.TestPlatform package or associated tools to enable code coverage gathering without requiring Visual Studio installation on the build agents.

fhnaseer commented 6 months ago

Can you please provide the following information?

  1. Testing.Platform nuget version
  2. Command line to collect code coverage
  3. Runsettings if any
  4. Can you please enable logs when collecting code coverage and share these logs with us?
ItaloCensi commented 6 months ago

Here the answers to your questions

Testing.Platform nuget version: 17.9.0 Command line to collect code coverage: please refer to the Testing.Platform.ps1 file in the attached zip Runsettings if any: please refer to the codecoverage.runsettings file in the attached zip Logs: please refer to the TestResults.trx file in the attached zip

Files.zip

fhnaseer commented 6 months ago

Can you please run vstest.console with /diag:log.vstest.txt and share log files? Also, can you please share which dlls is missing in coverage report? I am assuming that missing dlls with its pdb is present in the $UTLibraryFilePath folder.

ItaloCensi commented 6 months ago

The Log files are in attachment The dll missing in the coverage report is: USNext.SystemEngine.AWSSystem.dll Yes, the dll with its pdb is present in the $UTLibraryFilePath folder.

Logs.zip

fhnaseer commented 6 months ago

From the logs, it is trying to load this USNext.SystemEngine.AWSSystem.dll from GAC folder instead of $UTLibraryFilePath folder. It also tries to find in OriginalLocation, but pdb is missing there. You may have registered your libs in GAC and hence when running from vstest.console, it loads from GAC folder, rather than local folder. Can you please check why it is happening?

LoggerBase.LoadModuleData[0]: started Path:C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.02c73aaed268ba263\USNext.SystemEngine.AWSSystem.dll PdbRederProvider.TryFindPdbPath: OriginalLocation, Path: D:\Platform Development Team\Main\Source\AWSSystem\SystemEngine\obj\Release\USNext.SystemEngine.AWSSystem.pdb, fileExists: False PdbRederProvider.TryFindPdbPath: AssemblyLocation, Path: C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.0__2c73aaed268ba263\USNext.SystemEngine.AWSSystem.pdb PdbRederProvider.TryFindPdbPath: AssemblyLocation, Path: C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.02c73aaed268ba263\USNext.SystemEngine.AWSSystem.pdb PdbRederProvider.GetPdbDataProvider: None, not found. LoggerBase.LoadModuleData[0]: Loading module id:C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.02c73aaed268ba263\USNext.SystemEngine.AWSSystem.dll_73728_2299878681, path:C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.0__2c73aaed268ba263\USNext.SystemEngine.AWSSystem.dll ReportGenerator.LoadModule: name:C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.02c73aaed268ba263\USNext.SystemEngine.AWSSystem.dll, skipReason:no_symbols, detailed reason: StaticStreamCreator.CreateStream: Creating SharedMemoryStream. streamId:76, identifier:70f96c4f-8f2b-47d5-b339-44464642ff73, bufferName:Global\CodeCoverage.MTM_9908c82f-dfea-4cc9-a52b-e94ed9913562.d70f96c4f-8f2b-47d5-b339-44464642ff73, bufferSize:6 PipeClient.SendEmptyModuleInstrumentedMessage[0] LoggerBase.LoadModuleData[0]: finished Path:C:\Windows\assembly\GAC_MSIL\USNext.SystemEngine.AWSSystem\1.0.0.0__2c73aaed268ba263\USNext.SystemEngine.AWSSystem.dll, time taken: 1

ItaloCensi commented 6 months ago

Thank you very much for your feedback. I was able to resolve this issue by removing the USNext.SystemEngine.AWSSystem.dll from the GAC. I have a question about whether there is a way to configure the codecoverage.runsettings file to prevent specific libraries from using the GAC during testing.

fhnaseer commented 6 months ago

Here are a few settings to handle assembly paths while calculating code coverage.

Name Values Default Description Example
SymbolsRestrictOriginalPathAccess True, False False Determines if looking for a .pdb file in the original debug directory. <SymbolsRestrictOriginalPathAccess>True</SymbolsRestrictOriginalPathAccess>
SymbolsRestrictReferencePathAccess True, False False Determines if looking for a .pdb file is allowed in the path where the .exe file is located. <SymbolsRestrictReferencePathAccess>True</SymbolsRestrictReferencePathAccess>
SymbolsRestrictDBGAccess True, False False Determines if looking for debug information is allowed from .dbg files. <SymbolsRestrictDBGAccess>True</SymbolsRestrictDBGAccess>
SymbolsRestrictSystemRootAccess True, False False Determines if searching for .pdb files is allowed in the system root directory. <SymbolsRestrictSystemRootAccess>True</SymbolsRestrictSystemRootAccess>
ModulePaths XML Include and exclude lists for module paths.
Empty "Include" clauses imply all; empty "Exclude" clauses imply none. Each element in the list is a regular expression (ECMAScript syntax). See /visualstudio/ide/using-regular-expressions-in-visual-studio. An item must first match at least one entry in the include list to be included. Included items must then not match any entries in the exclude list to remain included.
<ModulePaths><Exclude><ModulePath>.*CPPUnitTestFramework.*</ModulePath></Exclude></ModulePaths>
SymbolSearchPaths XML Additional paths to search for .pdb (symbol) files. Symbols must be found for modules to be instrumented. If .pdb files are in the same folder as the .dll or .exe files, they are automatically found. Otherwise, specify them here. Note that searching for symbols increases code coverage runtime. So keep this small and local. <SymbolSearchPaths><Path>C:\Users\User\Documents\Visual Studio 2012\Projects\ProjectX\bin\Debug</Path></SymbolSearchPaths>

However, we cannot control from where a library is being loaded. So if your application is loading from GAC, you can provide SymbolSearchPaths where matching pdb can be found.

You can find all settings here. https://github.com/microsoft/codecoverage/blob/main/docs/configuration.md?plain=1