multiple Buildalyzer.Tests unit tests are failing

[Bertk]

dotnet CLI (Visual Studio 2022 preview 17.7.0 Preview 4.0) was used to build solutions from folders tests/repos/cake and tests/repos/Statiq.Framework. Some tests are failing because the unsupported .NET frameworks are not available.

Observations:

Warning As Error: Package 'SharpZipLib' 1.2.0 has a known moderate severity vulnerability, https://github.com/advisories/GHSA-m22m-h4rf-pwq3Warning As Error: Package 'SharpZipLib' 1.2.0 has a known low severity vulnerability, https://github.com/advisories/GHSA-mm6g-mmq6-53ffProcess 3596 exited with code 1

C:\Program Files\dotnet\sdk\8.0.100-preview.4.23260.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5): warning NETSDK1138: The target framework 'netcoreapp3.1' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/dotnet-core-support for more information about the support policy. [C:\GitHub\Buildalyzer\tests\repos\Statiq.Framework\tests\core\TestConsoleApp\TestConsoleApp.csproj]

Expected results: tests are executed successfully and do not require installation of unsupported .NET frameworks

Additional information

• https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-framework
• https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-core

[daveaglick]

Expected results: tests are executed successfully and do not require installation of unsupported .NET frameworks

Hmm, not sure that should be an expectation. Buildalyzer is essentially a runner for MSBuild, and the unit tests are designed to ensure that Buildalyzer is calling MSBuild correctly. The only way to do that is to actually call MSBuild, and since Buildalyzer currently supports calling MSBuild for those legacy .NET Framework projects, it stands to reason that they should be tested (that's sort of the whole sales pitch - it knows how to trigger a build, instrument that build with a logger, and analyze the build output of any .NET project, including older ones). These tests can probably be seen more as integration tests than unit tests since Buildalyzer itself is actually pretty thin once you get past shelling out to MSBuild correctly.

To put it another way, Buildalyzer supports calling MSBuild for legacy .NET versions, so the tests should too. That does mean they're not all runnable by every system, but that's okay. I've used open source projects as the subject of these tests because they seem to be complex enough to present problems like these, though there's nothing wrong with just commenting out that set of tests if you're just trying to see if Buildalyzer is working or want to make and test a small change.

Did I understand the problem statement correctly, or is this something totally different?

[Bertk]

I am a little bit overwhelmed with the long list of unsupported frameworks. 1) I feel tests for obsolete frameworks should be dropped if the functionality is already tested and it would be nice to use only supported frameworks for functionality verification. 2) The new .NET version 8.0 generates a warning for nuget packages which has a known vulnerability e. g. SharpZipLib 1.2.0. The build will not be successful. I used CLI dotnet test command.

By the way, until now I did not install framework 4.6.2 targeting pack on my desktop system. The preconditions for test execution are not documented.

[Bertk]

Today I checked the git status again.

This explains the reason and symptoms: https://gist.github.com/claraj/e5563befe6c2fb108ad0efb6de47f265

My resolution is a little bit different. I will delete the my clone/fork of the git repository.