dotnet / jitutils

MIT License
144 stars 59 forks source link

MLCSE: lay groundwork for optimizing for size #399

Closed AndyAyersMS closed 6 months ago

AndyAyersMS commented 6 months ago

Add a new --optimizeSize option where PolicyGradient tries to reduce code size instead of perf score.

Since there are now two possible optimization objectives, we have the potential for trading off one for the other. Update MCMC to track the "pareto frontier" for the methods it explores and save the data for visualization.

Contributes to dotnet/runtime#92915.

AndyAyersMS commented 6 months ago

@EgorBo PTAL cc @dotnet/jit-contrib

Example of a pareto frontier below. This shows the "best" size/score combinations achievable with CSEs, relative to the current JIT heuristic.

image

Note these "curves" must pass through or below (1,1); this point represents the current JIT heuristic behavior.

Here it passes through, meaning that we can either have smaller code or lower perf score than the current JIT, but not both. The lines joining the points are fictional as the observations are discrete, but they help visualize the nature of tradeoff. Also note we're at the mercy of MCMC's exploration strategy; it may be we should be doing more extensive random sampling and a more thorough exploration would change the shape of the curve in interesting ways. Will have to experiment some here.

Here's a case where the curve is below (1,1). For this method reducing code size seems "easy" but just one CSE sequence both reduces size and perf score.

image

AndyAyersMS commented 6 months ago

This needs a few more fixes since I changed all the ratios to divide by the baseline (meaning lower than 1.0 is now better in the geomeans, etc) but did not update the scoring logic... hope to push them soon.

FYI @NoahIslam @LogarithmicFrog1

AndyAyersMS commented 6 months ago

@EgorBo ping