[jvm] Fatal error: exception IO.Overflow("write_ui16")

[aduros]

One of our projects causes a compiler crash when targeting jvm:

Fatal error: exception IO.Overflow("write_ui16")
Raised at file "IO.ml", line 669, characters 36-59
Called from file "src/generators/jvm/jvmGlobals.ml", line 160, characters 22-50
Called from file "array.ml", line 77, characters 31-48
Called from file "src/generators/jvm/jvmAttribute.ml", line 221, characters 2-83
Called from file "list.ml", line 55, characters 20-23
Called from file "src/generators/jvm/jvmMethod.ml", line 1110, characters 19-100
Called from file "src/generators/jvm/jvmMethod.ml", line 1125, characters 14-35
Called from file "src/generators/jvm/jvmClass.ml", line 189, characters 25-50
Called from file "list.ml", line 73, characters 12-15
Called from file "src/generators/jvm/jvmClass.ml", line 185, characters 2-243
Called from file "src/generators/genjvm.ml", line 2515, characters 11-53
Called from file "src/generators/genjvm.ml", line 2064, characters 2-6
Called from file "list.ml", line 73, characters 12-15
Called from file "std.ml", line 26, characters 6-9
Re-raised at file "std.ml", line 28, characters 29-30
Called from file "src/generators/genjvm.ml", line 2919, characters 1-82
Called from file "src/compiler/haxe.ml", line 342, characters 2-14
Called from file "src/compiler/haxe.ml", line 1073, characters 25-72
Called from file "src/compiler/haxe.ml", line 633, characters 3-11
Called from file "src/compiler/haxe.ml", line 1238, characters 1-35

I can't share the whole project but I could spend some time trying to isolate a repro case if it would help. It doesn't seem to be using macros/abstracts or anything too crazy, although there is a bunch of level data defined in an array literal about 1000 lines long.

Thanks!

[Simn]

That fails while writing out the LineNumberTable. The specification says that it's alright to have multiple of those, so I guess I can just split them if they get too big.

However, you're likely going to hit some 16bit JVM limit in other places and I don't know if all of them can be worked around.

[Simn]

Actually this isn't going to help because the actual values are still 16bit, so any line or code offset beyond that would still error.

In fact, from reading a bit more it looks like there's a hard 16bit limit for code in general.

[Simn]

macro function make() {
    var num = 21845;
    var acc = [for (_ in 0...num) macro call()];
    return macro $b{acc};
}

@:pure(false)
function call() {}

function main() {
    make();
}

Error: LinkageError occurred while loading main class _Main.Main_Fields_
        java.lang.ClassFormatError: Invalid method Code length 65536 in class file _Main/Main_Fields_
Error: Command failed with error 1

It's a bit strange because the code itself has u4 code_length, but everything related to program counters is u2.

[Simn]

I don't consider this a bug, it's a limitation. The only course of action here would be to somehow split up functions that are too big, but that would come with a lot of complexity. For starters, even detecting this ahead of time is a big issue because we only know that we overflow at a low-level, by which point the overall transformation us already well underway. And even if we detected this, dealing with it while retaining the desired state and control flow would be quite complicated.

So while I'll look into improving the error message, this is something that has to be addressed in user code.

I'm curious if other compile-to-jvm languages try to resolve this somehow.

[aduros]

Ok, no worries :) Thanks for looking into it.

[PXshadow]

I'm having a similar issue but unrelated (at least indirectly) to function bodies/exprs

I created a Test typedef in stdgo/time/Time.hx:

typedef Test = Int;

then referenced the type:

function main():Void {
    var _x:stdgo.time.Time.Test;
}

Which results in the error: IO.Overflow("write_ui16")

Haxe version: 4.3.0-rc.1 hxjava: 4.2.0

[PXshadow]

I think this is actually a simpler problem in practice, then the theoretical one, of what if function X compiles into function Y is over the limit. In almost all cases the limit will be passed by a series of variables being initiated.

What if the initialization took place in a series of methods, i.e one method per variable, if that would be a large performance hit, perhaps then set a threshold of large variable initialization taking place in methods only?

[Simn]

I don't understand what you're saying. You have to initialize local variables locally because they're local variables.

[PXshadow]

I mean to say for static variables. I agree that the case for local variables is outside of the scope of Haxe, but static variables is still on the table I would imagine.

[Simn]

Ah, that could be achievable indeed. But is that actually what you're running into?

[PXshadow]

I can not see the same detailed error as you were seeing but I am almost positive, 6450 lines of static variable initialization vs 381 lines for everything else in the pretty isolated Unicode package (feel free to scroll and take a look at beautiful code generation).

Perhaps the original issue was also because of static variable initialization "bunch of level data defined in an array literal about 1000 lines long."

All of the initializer code adds up together where as the counting inside each method is isolated.

[PXshadow]

Speaking of, how would I go about getting the same detailed error as this?:

Error: LinkageError occurred while loading main class _Main.Main_Fields_
        java.lang.ClassFormatError: Invalid method Code length 65536 in class file _Main/Main_Fields_
Error: Command failed with error 1