jakubch1
commented
4 years ago Script that you can use:
`$src = "C:\git\roslyn\src"
$projects = New-Object System.Collections.ArrayList $nocc = New-Object System.Collections.ArrayList $coverlet = New-Object System.Collections.ArrayList $dynamic = New-Object System.Collections.ArrayList
function PerformTest($projectpath) { Write-Output "Tests for: " $project = [System.IO.Path]::GetFileNameWithoutExtension($projectpath) $dir = (Split-Path $projectpath -Parent) $projectpath
install_coverlet($dir)
$bin = "C:\\git\\roslyn\\artifacts\\bin\\$project\\Debug\\netcoreapp3.1"
if(Test-Path $bin)
{
$projects.Add($project)
run_no_cc $dir $bin $project
run_coverlet_cc $dir $bin $project
run_dynamic_cc $bin $project
Write-Output "Projects: $projects"
Write-Output "No cc: $nocc"
Write-Output "Coverlet: $coverlet"
Write-Output "Dynamic: $dynamic"
}}
function install_coverlet($dir) { Write-Output "Installing coverlet: " + $dir cd $dir dotnet add package coverlet.collector dotnet build dotnet publish --framework netcoreapp3.1 Write-Output "Coverlet installation finished: " + $dir }
function run_no_cc($dir, $bin, $project) { cd $dir
$sw = [Diagnostics.Stopwatch]::StartNew()
dotnet vstest "$bin\\$project.dll"
$sw.Stop()
$nocc.Add($sw.Elapsed) | Out-Null}
function run_coverlet_cc($dir, $bin, $project) { cd $dir
$sw = [Diagnostics.Stopwatch]::StartNew()
dotnet vstest "$bin\\$project.dll" --collect:"XPlat Code Coverage"
$sw.Stop()
$coverlet.Add($sw.Elapsed) | Out-Null}
function run_dynamic_cc($dir, $project) { cd $dir
if(Test-Path 'publish')
{
Remove-Item 'publish' -Recurse
}
$sw = [Diagnostics.Stopwatch]::StartNew()
vstest.console.exe "$project.dll" /Settings:"C:\tools\dotnet\coverage.runsettings"
$sw.Stop()
$dynamic.Add($sw.Elapsed) | Out-Null}
run_dynamic_cc($bin)
cd $src Get-ChildItem -Recurse *UnitTests.csproj | ForEach-Object { PerformTest($_.FullName) } `
MarcoRossignoli
github-actions[bot]
I was running performance comparison for roslyn tests projects. Results below. We were investigating why coverlet is much slower. You can clone https://github.com/dotnet/roslyn and retry this testing.
Few things that can be changed to improve performance: 1) initial instrumentation can be done in parallel for assemblies 2) review logic for probes and make minimal operations (definitely something is wrong here as Microsoft.CodeAnalysis.CSharp.Semantic.UnitTests took 32 minutes and normal run 8 minutes. In this case instrumentation took only 1 minute) 3) report generation could first generate stuff in parallel for all assemblies (in memory) then it can push data to report.