dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.42k stars 4.76k forks source link

Runtime should have a source-generated, trim/AOT-compatible XUnit runner #78342

Open agocke opened 2 years ago

agocke commented 2 years ago

By XUnit runner, I mean that there should be a library which allows a set of user-authored XUnit tests to be run that's substantially identical to the existing XUnit console runners.

Ideally, running an existing assembly of XUnit unit tests via the generated runner should have the same results as running it under the existing XUnit console runner.

Restrictions are:

The desired end state is that the runtime repo can rely on exactly one runner test base for all its unit testing in CI.

Non-goals:

I believe the existing XUnitWrapper project is a good starting point for this work, but it currently doesn't support conventional XUnit testing. We will need to add those capabilities.

ghost commented 2 years ago

Tagging subscribers to this area: @hoyosjs See info in area-owners.md if you want to be subscribed.

Issue Details
By XUnit runner, I mean that there should be a library which allows a set of user-authored XUnit tests to be run that's substantially identical to the existing XUnit console runners. Ideally, running an existing assembly of XUnit unit tests via the generated runner should have the same results as running it under the existing XUnit console runner. Restrictions are: - [ ] Tests must be runnable as a self-contained executable, to support the AOT form-factor - [ ] Test runner code must be trimmable - [ ] Runner must not depend on any non-trimmable, non-AOT compatible libraries The desired end state is that the runtime repo can rely on exactly one runner test base for all its unit testing in CI. Non-goals: - [ ] New runner supports end-user scenarios like `dotnet test` or VS test. If the semantics of the runner are the same, existing runner technologies can be used for these scenarios. I believe the existing XUnitWrapper project is a good starting point for this work, but it currently doesn't support conventional XUnit testing. We will need to add those capabilities.
Author: agocke
Assignees: -
Labels: `area-Infrastructure-coreclr`
Milestone: 8.0.0
jkoritzinsky commented 2 years ago

Here's a non-exhaustive list of features we are missing today in the XUnitWrapperGenerator's XUnit support that dotnet/runtime or other projects in the .NET product like ASP.NET Core use: