search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.43k stars 4.76k forks source link

Native AOT Testing Improvements #79316

Closed agocke closed 10 months ago

agocke commented 1 year ago
dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost commented 1 year ago

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas See info in area-owners.md if you want to be subscribed.

Issue Details
- [ ] https://github.com/dotnet/runtime/issues/72419 - [ ] https://github.com/dotnet/runtime/issues/67743 - [ ] https://github.com/dotnet/runtime/issues/67741
Author: agocke
Assignees: -
Labels: `area-NativeAOT-coreclr`
Milestone: -
kant2002 commented 1 year ago

I would like to ask opinions here first. Currently other teams like EF Core and ASP.NET seems to be plan to work on NativeAOT. From what I know they are using xUnit. Both does not have infra for NativeAOT yet.

I would like to ask what's status of testing of NativeAOT using xUnit, for example. If I want contribute to EF Core + NativeAOT for example, I obviously may try to create sample apps and work, but that seems to be inefficient.

It would be good to start discussing on guidance for developers who would like to tests their libraries.

MichalStrehovsky commented 1 year ago

In general, the experience we're shooting for with trimming and AOT is that of compile time warnings. If there are no warnings, the code should work and there's no need to do separate functional testing with trimming or AOT. The behaviors should be identical, modulo bugs in the runtime/compiler. It's the same for e.g. PublishReadyToRun. One doesn't specifically run unit tests with PublishReadyToRun either.

This only breaks down if one has a trimming/AOT incompatible dependency that the analyzer doesn't see. One could say that the testing would make sense to cover those. But the act of placing the code into a unit test makes trimming/AOT work differently (because the surrounding code is different). It may well be that the code will work in a test, but fail in the app, or vice versa. Using unit testing as a way to verify trimming/AOT compatibility doesn't work.

That said, we have #78342 tracking a trimming/AOT friendly xUnit runner for the purposes of the runtime repo. The runtime repo does run unit tests with NativeAOT, but the purpose is to catch actual NativeAOT runtime or compiler bugs. We're already using xUnit with its reflection based runner to run 200+ libraries test assemblies in this repo. But the experience is subpar and we have many rd.xml files to make things work in the runner.

agocke commented 10 months ago

Closing this out as done for 8.0. Further testing improvements will be brought through in 9.0