dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.85k stars 4.62k forks source link

Some of NativeAOT tests are disabled after sharing test cases with ILLink #82447

Open tlakollo opened 1 year ago

tlakollo commented 1 year ago

With https://github.com/dotnet/runtime/pull/82216 we started to share the test cases between ILLink and NativeAOT this allows us to delete duplicated code and keep tests consistent, given that a change in one of the tests will affect both codebases. Not all the ILLink tests apply to NativeAOT and that's why NativeAOT only enables a subset of them in its TestDatabase file, the test database file enables testing in a folder fashion, a folder is assigned to a feature being tested. But even in these features NativeAOT has not triaged/enabled all of the tests, and some of the tests inside a feature might not apply to NativeAOT. For these reasons we are disabling the test that have not been triaged inside a feature until they are triaged for:

Incomplete list of features which are still missing/tests:

ghost commented 1 year ago

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas See info in area-owners.md if you want to be subscribed.

Issue Details
With https://github.com/dotnet/runtime/pull/82216 we started to share the test cases between ILLink and NativeAOT this allows us to delete duplicated code and keep tests consistent, given that a change in one of the tests will affect both codebases. Not all the ILLink tests apply to NativeAOT and that's why NativeAOT only enables a subset of them in its [TestDatabase file](https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/aot/Mono.Linker.Tests/TestCases/TestDatabase.cs), the test database file enables testing in a folder fashion, a folder is assigned to a feature being tested. But even in these features NativeAOT has not triaged/enabled all of the tests, and some of the tests inside a feature might not apply to NativeAOT. For these reasons we are disabling the whole test while it gets triaged for: - Annotating the expectations of the test. Sometimes the test behavior is just different for NativeAOT and is expected. - A particular bug is fixed - A new feature is implemented - We determine that the feature doesn't apply to NativeAOT
Author: tlakollo
Assignees: -
Labels: `untriaged`, `area-NativeAOT-coreclr`
Milestone: -
MichalStrehovsky commented 1 year ago

@vitek-karas is there something we need for 8.0 or can we move this to 9.0?

vitek-karas commented 1 year ago

No - moved it to 9.0.