NativeUnpack Not Implemented

[TheLastRar]

IKVM version : IKVM-8.5.2-image-netcoreapp3.1-win7-x64

With vanilla Minecraft mostly working, I wanted to see if modded Minecraft would work, however, forge 10.13.4 uses Pack200. IKVM currently does not support Pack200.

I don't know if this is something that would be best included as a native library, or ported to C#

Take the following example code below, or the code listed here https://docs.oracle.com/javase/8/docs/api/java/util/jar/Pack200.html

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.jar.JarFile;
import java.util.jar.JarOutputStream;
import java.util.jar.Pack200;
import java.util.jar.Pack200.Packer;
import java.util.jar.Pack200.Unpacker;

public class Pack200Test {
    public static void main(String[] args) {
        // Create the Packer object
        Packer packer = Pack200.newPacker();

        try {
            JarFile jarFile = new JarFile("test.jar");
            FileOutputStream fos = new FileOutputStream("test.pack");
            // Call the packer
            packer.pack(jarFile, fos);
            jarFile.close();
            fos.close();

            File f = new File("test.pack");
            FileOutputStream fostream = new FileOutputStream("testExtracted.jar");
            JarOutputStream jostream = new JarOutputStream(fostream);
            Unpacker unpacker = Pack200.newUnpacker();
            // Call the unpacker
            unpacker.unpack(f, jostream);
            // Must explicitly close the output.
            jostream.close();
        } catch (IOException ioe) {
            ioe.printStackTrace();
        }
    }
}

You will get the following exception

Exception in thread "Thread-0" cli.System.NotImplementedException: The method or operation is not implemented.
        at IKVM.Java.Externs.com.sun.java.util.jar.pack.NativeUnpack.setOption(NativeUnpack.cs:60)
        at com.sun.java.util.jar.pack.NativeUnpack.setOption(Native Method)
        at com.sun.java.util.jar.pack.NativeUnpack.copyInOption(NativeUnpack.java:174)
        at com.sun.java.util.jar.pack.NativeUnpack.run(NativeUnpack.java:192)
        at com.sun.java.util.jar.pack.NativeUnpack.run(NativeUnpack.java:247)
        at com.sun.java.util.jar.pack.UnpackerImpl.unpack(UnpackerImpl.java:138)
        at com.sun.java.util.jar.pack.UnpackerImpl.unpack(UnpackerImpl.java:174)
        at packtest.Pack200Test.main(Pack200Test.java:32)
        at java.lang.reflect.Method.invoke(Method.java:486)
        at IKVM.Runtime.Accessors.Java.Lang.Reflect.MethodAccessor.InvokeInvoke(MethodAccessor.cs:35)

[wasabii]

This is a lot more possible now thank it was last week, now that we have clang projects in. I'll take a look.

[wasabii]

So, progress.

libunpack is OLD. And nasty.

Got it building though. However, it does some nasty trick where it attempts to load the JNI interface outside the JNI environment, scan for existing JVMs, and pick up an JNIEnv* from that. Instead of saving the JNIEnv that was handed to it by the JNI call itself.

This requires that we have JNI_GetCreatedJavaVMs available as something we can link to, and that can be resolved at runtime. We don't currently. Basically, a hosting API. It's real gross, and it would be pretty trivial to just fix in the C code I think, but I don't want to do that.

https://github.com/ikvmnet/jdk8u/blob/ebc5e190ac7a3b0c143451105d9f9b7d9e780a4b/jdk/src/share/native/com/sun/java/util/jar/pack/jni.cpp#L126

So, I need to get something of a libjvm.so working, and ensure it's loaded at runtime. That'll probably take a little bit.

[wasabii]

For history: IKVM used to have a jvm.dll file that was produced by the built. However, it was a Windows-only thing, produced by using implib to export the .NET functions out into a library.

The new approach will just be to have a hand done jvm shared library, written in C, that exports the three required functions. Only the one needed for unpack will be implemented today. That one, JNI_GetCreatedJavaVMs, will jump into a managed function provided by IKVM, and set by IKVM.Runtime at JNI initialization time. If not initialized, it will behave as if no VMs exist.

The other two functions are hosting related: JNI_GetDefaultJavaVMInitArgs and JNI_CreateJavaVM. I would like to support these eventually, but they're going to be pretty unlikely to be of much benefit to IKVM users. They allow you to spawn the JVM from outside the JVM. So, in IKVM land, they actually have to be wrappers around the various .NET hosting APIs for Framework and Core. So, hard to do, and of limited value. But, someday.

And for unpack to have a P2P to jvm, we'll need ProjectReference support during linking on IKVM.Clang.Sdk.

[TheLastRar]

I had hoped it would be relatively simple to add as a native library, but it seems like that wasn't the case given it's dependency on libjvm

However, it was a Windows-only thing, produced by using implib to export the .NET functions out into a library.

Would be nice if there was an equivalent on other platforms, although I'm not sure how feasible that would be from mono or netcore's perspective.

[wasabii]

The closest they get is the native aot stuff for exporting functions.

[wasabii]

Progress: IKVM.Clang.Sdk now supports P2P references. libjvm added, libjava added (missing bits), libverify added, libunpack added.

Close to seeing if it works.

[wasabii]

[wasabii]

So, commentary.

Ultimately pack200 is not a high priority for IKVM. It's been deprecated post JDK8. It was removed in JDK14. It's not an important library that any users other than this Minecraft thing care about. It'll eventually be removed in IKVM if we ever get to JDK14.

But, it served as an excuse to get the Clang builds of OpenJDK stuff working smoothly. And that has huge ramifications. So, yay.

[TheLastRar]

I did expect it would be a lower priority, so I've been impressed with the amount of effort you've put into this issue.

Nice to hear that this has led to more general improvements aswell