[jvm] Compiler failure when attempting to relocate `haxe.root.Array`

[EliteMasterEric]

Background

I am currently working on a Haxe library to apply shading to a targeted package or module. The modules the macro is applied to should be generated at a new location of the user's choice.

For example, --macro haxe.shade.Shade.applyPackage("test", "foo.bar") will cause src/test/CoolClass.hx to generate as foo.bar.CoolClass, rather than test.CoolClass.

This is being developed for a Minecraft modding project I am working on, where, if multiple JARs contain conflicting classpaths, the application will fail to load. Since this includes any modules in the haxe package, this has been a focus of my development for this library.

The module functions by applying the @:native metadata to the class, and removing any existing instances of that metadata to avoid overlap.

Note this library was originally in the form of a Haxe feature proposal from April of this year. The library is now in development after my project's switch to the JVM target (made possible by the resolution of #11023) mandated that shading no longer be done by Gradle.

The Issue

I am trying to move the haxe.root.Array module such that it is generated in the location of my choice, say foo.bar.haxe.Array.

Upon applying the macro, I receive this error:

Compiler failure
Please submit an issue at https://github.com/HaxeFoundation/haxe/issues/new
Attach the following information:
Haxe: 4.3.1; OS type: windows;
File "src/generators/genjvm.ml", line 2028, characters 11-18
Called from file "src/generators/genjvm.ml", line 1858, characters 3-13
Called from file "src/generators/genjvm.ml", line 2064, characters 5-15
Called from file "src/generators/genjvm.ml", line 2068, characters 3-10
Called from file "src/generators/genjvm.ml", line 2492, characters 3-21
Called from file "list.ml", line 110, characters 12-15
Called from file "list.ml", line 110, characters 12-15
Called from file "src/generators/genjvm.ml", line 2593, characters 2-49
Called from file "src/generators/genjvm.ml", line 2631, characters 2-63
Called from file "list.ml", line 110, characters 12-15
Called from file "src/core/timer.ml", line 199, characters 11-14
Called from file "src/generators/genjvm.ml", line 3079, characters 1-54
Called from file "src/compiler/generate.ml", line 99, characters 2-14
Called from file "src/compiler/compiler.ml", line 332, characters 29-64
Called from file "src/compiler/compiler.ml", line 347, characters 1-5
Called from file "src/compiler/compiler.ml", line 434, characters 2-162
Called from file "src/compiler/compiler.ml", line 408, characters 2-9
Called from file "src/compiler/compiler.ml", line 630, characters 5-43
Called from file "src/compiler/compiler.ml", line 640, characters 13-22
Called from file "src/compiler/server.ml", line 979, characters 1-39
Called from file "src/compiler/haxe.ml", line 53, characters 0-56
File "src/core/globals.ml", line 156, characters 1-7: Assertion failed

Reproduction

Download the library, use haxelib dev to install as a library, then compile ./test/issue-11361/compile.hxml, and see the error.

[EliteMasterEric]

My guess is that the core issue relates to an assumption somewhere that haxe.root.Array is where the class will be located.

However, since I can't apply shading after compilation and I can't NOT apply shading (or else the application will simply fail to start when trying to load two Minecraft mods created with my mod framework at the same time), it needs to be done during compilation.

Since reworking the code for the case where this specific annotation breaks things, a general purpose argument to shade the haxe package might be helpful.

Will note here that several cases where I attempted to apply my shading failed, including several typedefs in my own classes, as well as haxe.jvm.ClosureDispatch, haxe.jvm.Function, and haxe.jvm.VarArgs, among others. These all had @:native applied to them, but the values were simply ignored. May warrant a new issue if it's not resolved as a side effect to this one.

[Simn]

Yes, haxe.root and haxe.jvm are hardcoded in the generator, so if you mess it up via @:native things are going to break. This also applies to the Type API which makes some assumptions in that regard.

I'm not sure how to approach this. Something like https://github.com/HaxeFoundation/haxe-evolution/pull/108 would be a very general solution, but then you could still use @:native to mess everything up.

[EliteMasterEric]

A feature like HaxeFoundation/haxe-evolution#108 would directly solve my problem, both the main one and the secondary ones found in my comment (while making my shading library obsolete as a consequence ehe, I don't mind tho).

Applying the @:native annotation to a core Haxe class would still break stuff, but I don't see any reason why you would do that if shading was a feature, it would just break. Basically that one meme where the guy shoves the metal rod into the spokes of his own bike.

[Simn]

I'm not sure if this is really in-scope of a compiler though. Shouldn't there be Java tools that handle this kind of stuff at bytecode-level? Haxe's output works well enough as a compilation unit, and anything beyond that gets very complicated very quickly.

The only reason I'm entertaining this idea at all is because it's already necessary to map the empty package to haxe.root. Generalizing this could be good, but at the same time it's a quite specific case which needs support in multiple areas. As mentioned, we need to deal with the cases in https://github.com/HaxeFoundation/haxe/blob/development/std/jvm/_std/Type.hx#L103 (look for haxe.root), so if anything I feel like a specific -D jvm.root=different.package would make sense here.

[EliteMasterEric]

The issues here are fairly specific to my use case but basically unavoidable. There are methods to load JARs without causing conflicts. One Minecraft modding framework seems to use this method, but the other one doesn't, and the latter is more popular and I am not in a position to change it.

I don't know of any tools which can perform shading on compiled classes. The one tool I was using previously was operating on the Java sources, and had many issues because it was basically performing a regex find-and-replace.

[EliteMasterEric]

A specific -D jvm.root=<package> might make sense, since Java is the only language I can think of where this type of classpath conflict can occur.

[EliteMasterEric]

I am curious as to how work on resolving this issue has been progressing, since it is a blocker on implementing support for Forge.

[Simn]

I'll have to look into implementing this. I'm not sure how the compiler should communicate to the run-time what the root path is. I guess it could generate some generic haxe.jvm.CompilerInfos class which has the information as static fields. That doesn't seem very elegant, but maybe it's effective.