microsoft / testfx

MSTest framework and adapter
MIT License
679 stars 246 forks source link

Zero tests ran - Text is green indicating success. #3178

Closed thomhurst closed 1 week ago

thomhurst commented 1 week ago

If 0 tests are run, there is the message "Zero tests ran" in the output.

This returns you an exit code of 8, so it's a failed command.

However the text is green, implying it's a success.

I would opt to make the text red for any non 0 exit codes.

image

Evangelink commented 1 week ago

Could you please provide a reproducer? I am not seeing the same behavior:

image

thomhurst commented 1 week ago

Weird. Green for me even with a dummy adapter.

Code:

using Microsoft.Testing.Platform.Capabilities.TestFramework;
using Microsoft.Testing.Platform.Capabilities;
using Microsoft.Testing.Platform.Extensions.Messages;
using Microsoft.Testing.Platform.Extensions.TestFramework;
using Microsoft.Testing.Platform.Builder;

var builder = await TestApplication.CreateBuilderAsync(args);
builder.RegisterTestFramework(_ => new Capabilities(), (_, _) => new DummyAdapter());
using var app = await builder.BuildAsync();
return await app.RunAsync();

internal class DummyAdapter() : ITestFramework, IDataProducer
{
    public string Uid => nameof(DummyAdapter);

    public string Version => string.Empty;

    public string DisplayName => string.Empty;

    public string Description => string.Empty;

    public Type[] DataTypesProduced => new[] { typeof(TestNodeUpdateMessage) };

    public Task<CloseTestSessionResult> CloseTestSessionAsync(CloseTestSessionContext context) => Task.FromResult(new CloseTestSessionResult { IsSuccess = true });

    public Task<CreateTestSessionResult> CreateTestSessionAsync(CreateTestSessionContext context) => Task.FromResult(new CreateTestSessionResult { IsSuccess = true });

    public Task ExecuteRequestAsync(ExecuteRequestContext context)
    {
        context.Complete();
        return Task.CompletedTask;
    }

    public Task<bool> IsEnabledAsync() => Task.FromResult(true);
}

internal class Capabilities : ITestFrameworkCapabilities
{
    IReadOnlyCollection<ITestFrameworkCapability> ICapabilities<ITestFrameworkCapability>.Capabilities
        => Array.Empty<ITestFrameworkCapability>();
}

I'm using Powershell inside Windows Terminal on Windows 11

Invoked with dotnet run

Evangelink commented 1 week ago

Ok I can reproduce the error but only on 1.3.0 so looks like we regressed something.

cc @MarcoRossignoli