PMI: options for using local .Net Core builds

[AndyAyersMS]

As part of dev inner loop workflow it will be common to want to run PMI on local .Net Core builds. Here are some initial ideas on various approaches we could take.

For diffing and correctness checking we would like to use a checked build of the jit. So we either need a jit build that is compatible with the host CLI, or else we need a way to invoke a simpler locally built runner (say corerun).

For robustness we initially will likely use DRIVEALL to run the locally built jit in a child process, so we can push past asserts.

Once a particular method becomes interesting (from an assert or asm diff) we can use PREPONE to look at just that method. Currently that does not use a child process. So perhaps we need a DRIVEONE to parallel DRIVEALL.

If we go the CLI route, we can use the instructions in coreclr for using local nuget packages. This perhaps is most simply done by cloning the project file and adding new entries for the runtime. Or we can publish and then patch. It might make sense to always run the jit under test as an altjit. This would mean the cross-arch and self-arch jitting workflows would be the same.

Benchmark Dot Net seemingly solves similar issues and it might be interesting to consider adopting their approach; this would conceptually allow cross-runtime diffing (say comparing .net core vs .net framework codegen, or .net vs mono).

We probably want custom complus settings to apply only to the methods jitted via PMI. Even if we prejit/crossgen PMI.exe there will still be some residual jitting from SIMD methods in the core library. So we might need an assembly-inclusive altjit filter (currently there's only an exclusion list).

[AndyAyersMS]

So here's one way we could do this:

• sync local tree to something that is compatible with the dotnet CLI we're using
• modify local build so we build the protojit by default or via some build option
• do the local build
• publish the PMI app with the right runtime
• copy the protojit into the publish directory
• configure the environment to run the altjit and dump whatever output we need
• run PMI
• filter out the methods that got jitted that are not of interest

This currently doesn't use DRIVEALL, but I suppose it could. If we did that it would be nicer to supply the altjit info to PMI and have it set the environment for PREPALL.

@BruceForstall thoughts?

Partial batch file showing the middle steps:

@setlocal

@rem use PMI with checked local jit
@rem requires building protojit (not built by default)

set PMI_ROOT=D:\repos\jitutils\src\pmi
set CORE_BIN=D:\repos\coreclr\bin\Product\Windows_NT.x64.Checked
set CLI_BIN="C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.0"

cd /d %PMI_ROOT%

@rem --- prepare test sandbox

dotnet publish -c Release --framework netcoreapp2.1 --runtime win-x64

set PMI_RUN_DIR=%PMI_ROOT%\bin\Release\netcoreapp2.1\win-x64\publish

@rem --- set up alt jit in sandbox, failure tolerant mode

set complus_altjitname=protojit.dll
copy %CORE_BIN%\%complus_altjitname% %RUN_DIR%

@rem --- configure runtime to use altjit, failure tolerant mode

set complus_altjit=*
set complus_altjitskiponassert=1
set complus_continueonassert=1

@rem --- try and avoid altjit running on assemblies that are used by PMI itself

set complus_altjitexcludeassemblies=System.Private.Corelib;pmi

@rem --- look at jit codegen

set complus_jitdisasm=*

%PMI_RUN_DIR%\pmi.exe PREPALL %1

[BruceForstall]

sync local tree to something that is compatible with the dotnet CLI we're using

I don't have much experience using the CLI, but this part seems problematic. It seems like we should be able to use any dotnet/coreclr build / commit hash without worrying about CLI compatibility (e.g., the flow of breaking changes between repos that drive CLI creation). Maybe that argues for building PMI as part of dotnet/coreclr and running it with corerun, so corerun, PMI, S.P.C.dll, and the JIT will be guaranteed to be in sync with a single build step.

I'm not sure I fully understand what the benefit of using altjit is.

[AndyAyersMS]

We don't need an exactly matching CLI -- just one that uses the same jit GUID.

Aside from the net45/netcoreapp2.1 distiction there's no fine-grained runtime dependence in PMI itself. We could easily run it via corerun (though driveall may possibly need more tweaking). It doesn't need to be built in concert with CoreCLR.

For example:

D:\repos\coreclr\bin\tests\Windows_NT.x64.Checked\Tests\Core_Root\corerun.exe D:\repos\jitutils\src\pmi\bin\Release\netcoreapp2.1\publish\PMI.dll PREPALL d:\bugs\18022\ex.exe
Prepall for d:\bugs\18022\ex.exe
Start type X
PREPALL type# 0 method# 0 X::MethodBar, elapsed time: 00:00:00.0003798, elapsed ms: 0.3798
PREPALL type# 0 method# 1 X::Main, elapsed time: 00:00:00.0004418, elapsed ms: 0.4418
PREPALL type# 0 method# 2 X::.ctor, elapsed time: 00:00:00.0003877, elapsed ms: 0.3877
Completed type X, type elapsed time: 00:00:00.0339004, elapsed ms: 33.9004
Completed assembly d:\bugs\18022\ex.exe - #types: 1, #methods: 3, elapsed time: 00:00:00.1159310, elapsed ms: 115.931
ERRORLEVEL=0

So an alternate innerloop path is just to create a bootstrap script or app that does just this, more or less.

Benefit of using altjits is that we don't have to destructively alter the publish dir (in the "CLI is driver" scenario) , can easily run crossjits, and can probably recover from most asserts w/o the extra DRIVEALL layering, and might have an easier time filtering out "baseline" method jitting.

[AndyAyersMS]

Initial support via #134 is using a local corerun under the covers.