[improvement] Removed all usage of reflection, enabled trimming on UWP, improved memory usage when compiling .NET Native

Arlodotexe commented 2 years ago

Overview

This PR attempts to fix an issue where .NET Native throws errors when building release on a Azure DevOps hosted agent.

We've been seeing various errors, mostly "Out of memory" and "Internal compiler error". This PR attempts to fix them.

Progress towards #170

Checklist

This PR meets the following requirements:

[x] Tested and contains NO breaking changes or known regressions.
[x] Tested with the upstream branch merged in.
[x] Tests have been added for bug fixes / features (or this option is not applicable)
[x] All new code has been documented (or this option is not applicable)
[x] Headers have been added to all new source files (or this option is not applicable)

Additional info

Not provided

Arlodotexe commented 2 years ago

If the Build Validation doesn't succeed on the first try then the fix was not successful. Once we get a successful run, I'll be running the build validation multiple times against this branch to make sure this is actually fixed.

Arlodotexe commented 2 years ago

Relevant error log:

Arlodotexe commented 2 years ago

It would appear that the problem has been slightly mitigated. The issue still appears sometimes, but it's much more likely to succeed than fail now, which is enough to usually pass the CI with a max of 3 retries.

Arlodotexe commented 2 years ago

I've also set the pipeline to never retry, so now we should know if it succeeded in less than 20 minutes

Arlodotexe commented 2 years ago

~~We've managed to fix the fatal error N1002: Out of Memory error.~~ (updated: see below)

But we're still pretty consistently getting error : ILT0005: and Error : Internal Compiler Error when the build does fail.

These might contain some useful information:

Arlodotexe commented 2 years ago

After 3 more runs, we've managed to get several additional errors. These all seem to always be paired to the existing ILT0005 message, never appearing on their own, sometimes not appearing at all.

This occured while scanning the methods for the MDIL compiler:

fatal error N1002: Out of Memory (TaskId:893)

These occurred after successfully generating the StrixMusic.mdilexe, still failing the GenerateMDILGlobalAnalysis task.

Error: NUTC1056:Internal Compiler Error: 0x8007000e. Error encountered

NUTC300F:Internal Compiler Error: Native compilation failed due to out of memory error

So far, in an attempt to curb these issues, we've enabled these properties:

    <Use64BitCompiler>true</Use64BitCompiler>
    <PreferredToolArchitecture>x64</PreferredToolArchitecture>

    <SingleThreadNUTC>true</SingleThreadNUTC>
    <UseDotNetNativeToolchain>true</UseDotNetNativeToolchain>
    <EnableGatekeeperAnalysis>true</EnableGatekeeperAnalysis>
    <ShortcutGenericAnalysis>true</ShortcutGenericAnalysis> <!-- Can help stop runaway analysis of generic types and reduce overall generation requirements. -->
    <OutOfProcPDB>true</OutOfProcPDB> <!-- Opt-in to generate PDB's out of process -->

It should be noted that Azure DevOps machines (where these are failing) only get 7Gb of RAM and 2 processor cores.

Arlodotexe commented 2 years ago

I've used this to create a repro so the .NET Native team can take a look: https://github.com/dotnet/core/blob/main/Documentation/ilcRepro.md

The files have been uploaded to ipfs:

Gateway link	CID
netnativerepro-ARM.zip	QmQpxWTHjmQon6HzBT8ymwbjqhpsLY2aQnApadQ6kd3Z7D
netnativerepro-x64.zip	QmavC7xzzQDQpeaknBz5Fs5DwpEpzey7JYJU5UvGR9TyZG
netnativerepro-x86.zip	QmcgFgsTFGmrDDpf7n6fjD9JyVG8HyM2sVBuFYw6t2chKF

Arlodotexe commented 2 years ago

We've done everything we possibly can, and we'll need to wait for the .NET Native team to get back to us before we can do more.

That in mind, this PR does contain a number of useful improvements that we'll want to keep. We'll be merging this in and closing this off, saving the full fix for later.

Arlodotexe / strix-music