0.14.0 cannot find tests (0.13.3 works fine)

[cr3wdayt5p]

Hi

I just tried to update from 0.13.3 til 0.14.0, and now my tests don't run with dotnet test.

I get the following message:

No test is available in [/path/to/my/src]/UnitTests/bin/Debug/net7.0/UnitTests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

My tests runs fine if I switch back to 0.13.3.

[Alxandr]

Which version of expecto are you using?

[cr3wdayt5p]

<PackageReference Include="Expecto" Version="10.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.2" />

When using 0.13.3 I get a warning, but tests still works as expected:

warning NU1608: Detected package version outside of dependency constraint: YoloDev.Expecto.TestSdk 0.13.3 requires Expecto (>= 9.0.0 && < 10.0.0) but version Expecto 10.0.0 was resolved.

[cr3wdayt5p]

I just tried to update to the newest .NET SDK. But that did not solve the issue (but 0.13.3 still works).

For reference here is my dotnet --info:

.NET SDK:
 Version:   7.0.305
 Commit:    98e1b6c381

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  13.4
 OS Platform: Darwin
 RID:         osx.13-arm64
 Base Path:   /usr/local/share/dotnet/sdk/7.0.305/

Host:
  Version:      7.0.8
  Architecture: arm64
  Commit:       4b0550942d

.NET SDKs installed:
  7.0.305 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  x64   [/usr/local/share/dotnet/x64]
    registered at [/etc/dotnet/install_location_x64]

Environment variables:
  Not set

global.json file:
  Not found

[Alxandr]

That's unfortunate. I might wanna yank 0.14 then. The weird thing is that it did work on CI though.

[Alxandr]

Have you tried a blank project? And could you try specifying the test adapter path?

[cr3wdayt5p]

I have now tested with this small blank project:

Blank.fsproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <GenerateProgramFile>false</GenerateProgramFile>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="Tests.fs" />
    <Compile Include="Main.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Expecto" Version="10.1.0" />
    <PackageReference Include="YoloDev.Expecto.TestSdk" Version="0.14.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.3" />
  </ItemGroup>
</Project>

Main.fs:

module Main

open Expecto

[<EntryPoint>]
let main argv =
    Tests.runTestsInAssemblyWithCLIArgs [] argv

Tests.fs:

module Tests

open Expecto

[<Tests>]
let tests =
  testList "samples" [
    test "pass1" { Expect.equal false false "" }
    test "pass2" { Expect.equal true true "" }
  ]

But same result – and 0.13.3 works.

How do I specify a path to a test adapter? And what should it point to?

[Alxandr]

It's been a while since I've done any development on this, but going off memory, you just need to point to the installed .dll in your .nuget folder. Example /TestAdapterPath:<pathToCustomAdapters>. I should also try to add .NET 7 to the test matrix to see if the issue is reproducible on CI.

[cr3wdayt5p]

Based on (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test) I got to this command:

dotnet test Blank.fsproj --test-adapter-path /Users/mbk/.nuget/packages/yolodev.expecto.testsdk/0.14.0/build/net6.0/expecto.visualstudio.dotnetcore.testadapter.dll

This gave the following result:

Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (arm64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) '/Users/mbk/local/blank-yolodev-expecto-test/Blank.fsproj' exited with error: You must install or update .NET to run this application.
App: /usr/local/share/dotnet/sdk/7.0.305/testhost.dll
Architecture: x64
Framework: 'Microsoft.NETCore.App', version '7.0.8' (x64)
.NET location: /usr/local/share/dotnet/x64/
The following frameworks were found:
  3.1.32 at [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App]
  6.0.12 at [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App]
  6.0.13 at [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App]
Learn about framework resolution:
https://aka.ms/dotnet/app-launch-failed
To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=7.0.8&arch=x64&rid=osx.13-x64
. Please check the diagnostic logs for more information.

Test Run Aborted.

I then installed the x64 version of the SDK as well.

Then I ran the exact same command again and got this result:

Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (arm64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) '/Users/mbk/local/blank-yolodev-expecto-test/Blank.fsproj' exited with error: Unhandled exception. System.BadImageFormatException: An attempt was made to load a program with an incorrect format.
 (0x8007000B)
. Please check the diagnostic logs for more information.

Test Run Aborted.

But I am not sure the --test-adapter-path approach is the right one as I cannot make it work with 0.13.3. In the .nuget/packages/yolodev.expecto.testsdk/0.13.3/build folder there is only net5.0 and netcoreapp3.1 builds and if I specify any of those with --test-adapter-path I get the same error as above.

dotnet test Blank.fsproj --test-adapter-path /Users/mbk/.nuget/packages/yolodev.expecto.testsdk/0.13.3/build/netcoreapp3.1/expecto.visualstudio.dotnetcore.testadapter.dll

dotnet test Blank.fsproj --test-adapter-path /Users/mbk/.nuget/packages/yolodev.expecto.testsdk/0.13.3/build/net5.0/expecto.visualstudio.dotnetcore.testadapter.dll

But just as before using the simple dotnet test command on version 0.13.3 works fine.

[Alxandr]

The fact that you got it working by installing a new version of .NET framework is very interesting... Does it still not work to just do dotnet test after you got it working with --test-adapter-path? Specifying the test adapter path is just a way to force .NET to use a specific test adapter, while the default is to have it automagically figure it out, hence it's valuable in troubleshooting.

[cr3wdayt5p]

No, 0.14.0 still does not work. Neither with dotnet test or dotnet test Blank.fsproj --test-adapter-path ....

Installing the x64 SDK only changed the error I got when specifying --test-adapter-path.

[cr3wdayt5p]

So to summarize I cannot run any tests with the --test-adapter-path flag.

• Sample project above (as is, i.e. 0.14.0):
  • dotnet test: fail
  • dotnet test Blank.fsproj --test-adapter-path [...]/0.14.0/build/net6.0/[...]: fail
• Modified sample project (only changed 0.14.0 to 0.13.3):
  • dotnet test: works
  • dotnet test Blank.fsproj --test-adapter-path [...]/0.13.3/build/net5.0/[...]: fail
  • dotnet test Blank.fsproj --test-adapter-path [...]/0.13.3/build/netcoreapp3.1/[...]: fail

[Numpsy]

I just had a quick test, and it seems like dotnet test will find and run the tests if I manually copy the test adaptor dll into the test project bin directory before running the tests.

I notice that the old version of the test adaptor had a .props file that copied expecto.visualstudio.dotnetcore.testadapter.dll into the output directory and the new one doesn't - is that an intentional change?

[cr3wdayt5p]

I can confirm that if I manually copy the test adapter dll into test project /bin/Debug/net7.0 then 0.14.0 works with dotnet test.

Also when I do dotnet test --verbosity detailed with 0.13.3 then I can see that it uses the netcoreapp3.1version of the dll.

[cr3wdayt5p]

I can also confirm that if I copy [...]/.nuget/packages/yolodev.expecto.testsdk/0.13.3/build/netcoreapp3.1/YoloDev.Expecto.TestSdk.props into [...]/.nuget/packages/yolodev.expecto.testsdk/0.14.0/build/net6.0/YoloDev.Expecto.TestSdk.props then I can successfully run dotnet test in the sample project (0.14.0).

So I guess the fix is to create this missing .props file?

[Alxandr]

Aha. That should be it then. It was probably an oversight of mine in the review of the PR upgraded to expecto 10 - and unfortunately CI does not catch this :-/. I'm not currently at a location where I have access to a dev environment for this, but if anyone would like to make a PR re-adding the file as src/YoloDev.Expecto.TestSdk/build/net6.0/YoloDev.Expecto.TestSdk.props I can create a release for it.

[cr3wdayt5p]

I tried to create this using only the Github GUI. It was doable :)

I added the reference in .fsproj as well as it was remove here in #120.

[Alxandr]

Ah - I didn't think about that option. I'll review it immediately.

[Alxandr]

v0.14.1 should be available on nuget.org in roughly 2 hours. Please let me know if the issue still persists :). Thanks for the help in debugging and resolving the issue.

[Numpsy]

The new version seems to work for me now (.NET 6 unit test project)

[cr3wdayt5p]

0.14.1 is also working just fine for my project :)