Tunable code size threshold

[rahulmutt]

The JIT is highly sensitive to size of methods that are generated. This means we should be wary of how much compile-time inlining is done. But at the same time, compile-time inlining enables a cascade of other optimizations making lazy functional programs extremely efficient. As a result, it is possible to get huge STG functions that end up generating methods that exceed the 65,535. While this doesn't happen with user-written code as observed so far, it does seem to happen with code generated by tools like alex and happy.

The way we get the best of both words is to add a new -fmax-code-size=[n] flag and a new step in CorePrep (the Core->STG pre-processing step) that: 1) Calculates approximate code size using a heuristic, taking into account how the code generator generates code. 2) Will generate new bindings in such a way that:

• The code size doesn't exceed the value provided subject to the constraints that:
  • Recursive calls aren't broken into new bindings.
  • Let-no-escape calls aren't broken into new bindings.

[jneira]

@rahulmutt Hi, i've detected that cborg library reach the limit:

Exception in thread "Thread-278" java.lang.ClassFormatError: Invalid method Code length 171742 in class file cborg/codec/cbor/Read
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$100(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at dhall.dhall.Import.exprFromImport1(Import.hs)
        at dhall.dhall.Import$exprFromImport1.apply3V(Import.hs)
        at eta.runtime.apply.PAPSlow.apply(PAPSlow.java:73)
        at eta.runtime.apply.PAPSlow.applyV(PAPSlow.java:119)
        at eta.runtime.thunk.Thunk.applyV(Thunk.java:121)
        at eta.runtime.exception.Exception.catch_(Exception.java:132)
        at dhall.dhall.Import.$La$1(Import.hs)
        at dhall.dhall.Import$$La$1.enter(Import.hs)
        at eta.runtime.apply.PAPSlow.apply(PAPSlow.java:113)
        at eta.runtime.apply.PAPSlow.apply1V(PAPSlow.java:197)
        at dhall.dhall.Import$sat$422.apply1V(Import.hs)
        at eta.runtime.thunk.Thunk.apply1V(Thunk.java:156)
        at dhall.dhall.Import$sat$409.apply1V(Import.hs)
        at main.Import$sat$8.applyV(Import.hs:73)
        at eta.runtime.exception.Exception.catch_(Exception.java:132)
        at main.Import$sat$7.applyV(Import.hs:76)
        at tasty_hunit.test.tasty.HUnit$sat$1.applyV(HUnit.hs:102)
        at eta.runtime.exception.Exception.catch_(Exception.java:132)
        at tasty_hunit.test.tasty.HUnit.$wa(HUnit.hs:92)
        at tasty_hunit.test.tasty.HUnit.$fIsTest_TestCase1(HUnit.hs)
        at tasty_hunit.test.tasty.HUnit$$fIsTest_TestCase1.apply3V(HUnit.hs)
        at eta.runtime.apply.PAPSlow.apply(PAPSlow.java:73)
        at eta.runtime.apply.PAPSlow.applyV(PAPSlow.java:119)
        at eta.runtime.thunk.Thunk.applyV(Thunk.java:121)
        at eta.runtime.exception.Exception.unmaskAsyncExceptions(Exception.java:94)
        at async.control.concurrent.Async$sat$246.apply1V(Async.hs)
        at async.control.concurrent.Async$sat$235.applyV(Async.hs)
        at eta.runtime.exception.Exception.catch_(Exception.java:132)
        at async.control.concurrent.Async$sat$234.applyV(Async.hs)
        at eta.runtime.stg.Closures$EvalLazyIO.enter(Closures.java:145)
        at eta.runtime.stg.Capability.schedule(Capability.java:254)
        at eta.runtime.concurrent.WorkerThread.run(WorkerThread.java:16)

Would be there a workaround like in #557?

[rahulmutt]

The workaround in #557 worked because alex generates code that we could post-process. But for cborg, it may involve turning off inlining or modifying the source of the library itself. There is a long-term solution to this problem and we will need to fix it asap since running Eta that was built with Eta fails with the same error.

The long-term solution seems to be adding an STG-to-STG pass that checks when the AST depth large and generates new local let bindings to cut down the depth. Would this be something you're interested in working on?

[jneira]

Yeah, but not sure i am going to have time to start right now :worried:

[jneira]

In the meanwhile i would like to patch cborg to avoid the issue: i started to convert INLINE to NOINLINE but i had no luck, what would be the required changes?

[rahulmutt]

You need to take a look at the particular method that had too many bytecodes and match that method with the name in STG. Then, a look at the STG code to figure out what exactly was inlined and add NOINLINE's to the corresponding source. This will require you to dig into the internals of cborg a bit.

[jneira]

Hi, at least i get to avoid the issue, use -ddump-inlining ghc flag was helpful too! https://github.com/typelead/eta-hackage/pull/135