No test is available message from dotnet test CLI on .NET core 3.1 on docker container

[vikassd2012]

Hi there,

Spoke earlier with Xunit team on slack including Brad Wilson about this issue. He suggested to file this issue with repro project.

We have a jenkin pipeline that runs our dotnet core tests : ENVIRONMENT: Docker container based of mcr.microsoft.com/dotnet/core/sdk:3.1 (3.1.301 dotnet --version)

Here is summary of issue:

1. While trying to execute our test from a test project we use dotnet test cli like this: dotnet test Services/MyService/build.proj /p:CollectCoverage=true /p:CoverletOutput=./TestResults/ /p:CoverletOutputFormat=cobertura /p:CopyLocalLockFileAssemblies=true --configuration Release --logger trx;LogFileName=ci_tests.trx --logger console;verbosity=detailed -v d --no-build

We have at least 10 other microservices that use this same way to execute tests and it works consistently. However we added a new service in our solution and this new service is giving below message while we try to run the tests:

No test is available in /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/bin/Release/netcoreapp3.1/UnitTestProject.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/TestResults/ci_tests.trx

However instead of this .proj file, if we execute the test directly like this: dotnet test tests/UnitTests/UnitTestProject/UnitTestProject.csproj It works.

2. To make things more interesting, then I tried following: a. Execute the dotnet test via : dotnet test tests/UnitTests/UnitTestProject/UnitTestProject.csproj Test works as earlier.

b. Then run the dotnet test again via the other command: dotnet test build.proj /p:CollectCoverage=true /p:CoverletOutput=./TestResults/ /p:CoverletOutputFormat=cobertura /p:CopyLocalLockFileAssemblies=true --configuration Release --logger 'trx;LogFileName=ci_tests.trx' --logger 'console;verbosity=detailed' -v d --no-build

But since I ran the step a, I noticed step b also works now. Its able to find the test and run the test.

3. Also this works consistently on our local both MacOS and Windows.

Attaching a sample application with which we were able to reproduce the issue. For Xunit Team.zip

STEPS TO REPRODUCE

1. Unzip the zip to a folder on

   linux machine (Debian 10) or 
   docker container running this docker image: mcr.microsoft.com/dotnet/core/sdk:3.1

2. Run the following command from command line: dotnet test build.proj /p:CollectCoverage=true /p:CoverletOutput=./TestResults/ /p:CoverletOutputFormat=cobertura /p:CopyLocalLockFileAssemblies=true --configuration Release --logger 'trx;LogFileName=ci_tests.trx' --logger 'console;verbosity=detailed' -v d --no-build

EXPECTED RESULTS:

3. We expect this:

   
   Microsoft (R) Test Execution Command Line Tool Version 16.6.0
   Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern. /Users/vyadav/Desktop/For Xunit Team/tests/UnitTests/UnitTestProject/bin/Release/netcoreapp3.1/UnitTestProject.dll [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.0 (64-bit .NET Core 3.1.5) [xUnit.net 00:00:00.36] Discovering: UnitTestProject [xUnit.net 00:00:00.40] Discovered: UnitTestProject [xUnit.net 00:00:00.41] Starting: UnitTestProject [xUnit.net 00:00:00.49] Finished: UnitTestProject √ UnitTestProject.UnitTest1.Test1 [2ms] Results File: /Users/vyadav/Desktop/For Xunit Team/tests/UnitTests/UnitTestProject/TestResults/ci_tests.trx

Test Run Successful. Total tests: 1 Passed: 1 Total time: 1.1717 Seconds

ACTUAL RESULTS:
4. We get this:

Test run for /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/bin/Release/netcoreapp3.1/UnitTestProject.dll(.NETCoreApp,Version=v3.1) Microsoft (R) Test Execution Command Line Tool Version 16.6.0 Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern. /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/bin/Release/netcoreapp3.1/UnitTestProject.dll No test is available in /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/bin/Release/netcoreapp3.1/UnitTestProject.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. Results File: /home/ec2-user/workspace/ipeline_els_jenkins-repro-simple/Services/ErrorLookupService/tests/UnitTests/UnitTestProject/TestResults/ci_tests.trx

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:.

ADDITIONAL NOTES:
5. We also needed to tweak our .csproj file to include following lines due to the reasons mentioned in the comments below:

    <IsTestProject>true</IsTestProject>
    <!--needed to add this otherwise was getting error Testhost process exited with error: A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found -->
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>

From 100+ test projects we have in solution, only few exhibit this issue and expect above 2 lines, but once we add those 2 lines, it works. But this new service along with this recent 3.1 net core update, its not working. But other services are just fine.

Please let me know if you have any questions.
Looking forward to finding a solution to this.

Regards,
Vikas

[bradwilson]

I cannot reproduce the problem with your ZIP file.

The two commands I ran after unzipping your repro project were:

dotnet build build.proj
dotnet test build.proj /p:CollectCoverage=true /p:CoverletOutput=./TestResults/ /p:CoverletOutputFormat=cobertura /p:CopyLocalLockFileAssemblies=true --configuration Release --logger 'trx;LogFileName=ci_tests.trx' --logger 'console;verbosity=detailed' -v d --no-build

This worked as expected on Ubuntu 18.04:

It also worked with the mcr.microsoft.com/dotnet/core/sdk:3.1 Docker image:

[vikassd2012]

Thanks @bradwilson for looking into this. This is strange why we are seeing this issue. I will compare your test environment and try to see if there any differences.

[bradwilson]

Closing this for age. My best recommendation would be to open this with the .NET team to understand why your hand-crafted .proj file does not work when a normal .csproj file does.