IKVM.Java can't be exported to a TLB

[jhuber1965]

Hello!

I am upgrading my application from Winward IKVM 8.5.0.2 to ikvm-revived/ikvm 8.2.3 and have run into the problem below. I am creating .NET DLLs with COM support from my java code. When I use REGASM to register my DLL for use with COM applications, REGASM crashes with the following error:

[...]
Type 'com.sun.tools.doclint.DocLint' exported.  
Type 'com.sun.tools.doclint.DocLint+BadArgs' exported.
Type 'com.sun.tools.doclint.Entity' exported.
RegAsm : error RA0000 : Type library exporter encountered an error while processing  
     'com.stanref.st.utilities.ComparatorGeneric, StanRef.ShellAndTube.utilities'. 
Error: Type library exporter encountered an error while processing  
     'com.sun.tools.doclint.Entity+__Enum.agrave, IKVM.Java'. 
Error: Ambiguous name.

If I open up IKVM.Java.dll with ILSpy, it seems that com.sun.tools.doclint.Entity__Enum has several entries like Agrave and agrave, Pi and pi, etc. While these are unique entries in Java, they are not unique entries for COM, and I think this is making REGASM crash with the "Ambiguous name" error. One can also see this in the java code at https://code.googlesource.com/edge/openjdk/+/refs/heads/jdk8u111-b09/langtools/src/share/classes/com/sun/tools/doclint/Entity.java

Is there any way I can work around this? I also tried to load my DLL directly from Excel, but it will not load the library. In the previous IKVM 8.5.0.2, I did not have this issue.

Thanks!!

[wasabii]

Well, so, assuming the issue is because previous releases of IKVM (Windward, etc) split the Java classes apart into multiple assemblies, so you just never hit this. And potentially didn't even include the tools.

I would be pretty hesitant to say that exporting the entirety of Java to COM objects is a good design decision, however. I probably would have developed some .NET code that exposes my own types, with only the requirements I need.

It's also been over 15 years since I've messed with regasm. So I'd wonder if there was a way to limit it to specific types/methods.

[jhuber1965]

Thank you for the reply! I don't think I am consciously trying to export the entirety of Java. In my Java code, I'm just annotating the classes I want to expose with

@cli.System.Runtime.InteropServices.ClassInterfaceAttribute.Annotation(cli.System.Runtime.InteropServices.ClassInterfaceType.__Enum.AutoDual)

To register the assembly, I'm issuing the following command:

"C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\regasm" StanRef.ShellAndTube.utilities.dll /tlb:StanRef.ShellAndTube.utilities.tlb  

If I use regasm to make a reg file only, all the reg file has in it are the classes I expose. I think the problem is making the type library (TLB file) for COM (the /tlb option).

If I look at the TLB file created when using regasm on the DLL created from previous IKVM, all that is in the TLB file are my classes; there are no IKVM or core Java classes, so something in the DLL created with the new IKVM must be triggering regasm to export all these core Java classes. I'm not an expert with regasm so I don't really know what is going on.

I may have to look at the tlbexp tool to create the TLB separately. It has the /oldnames option for decoration when name conflicts occur.

Otherwise, I guess I will just have to stick with the old IKVM, as it does still seem to run fine with my code.

[wasabii]

So the docs for regasm says it follows all references, generating multiple tlbs. And it says you can't export a partial assembly. And /oldnames does seem to be a thing that might work.

Is your custom assembly exposing java types? If not, tlbexp will probably work.

[jhuber1965]

My assembly is part of a calculation engine that our customers include into their software. The calculation engine is written in Java, and has to support clients that use Java, .NET, and COM. Thus, my assembly created with IKVM exposes types and classes to COM.

I think regasm basically calls tlbexp. I tried tlbexp with /oldnames, and it crashes with the same error above. I also ran tlbexp directly on IKVM.Java.dll with /oldnames, and it crashes with the above error.

In any case, I believe that your above suspicion about the previous IKVM having the java classes split across multiple assemblies is correct. With old IKVM, in order to get it to work with COM, it was necessary to create/register tlb files for three IKVM DLLs: IKVM.OpenJDK.Core.dll, IKVM.OpenJDK.Security.dll, and IKVM.OpenJDK.Util.dll. com.sun.tools.doclint is in IKVM.OpenJDK.Tools.dll, and since it was not necessary to create a TLB file for that before, I never ran into this problem.

Now with most everything combined into IKVM.Java.dll, com.sun.tools.doclint has to get registered with COM, but it can't, because of the naming conflict. Also, a whole bunch of other classes that don't need to be registered with COM now would get registered as well. With the current setup, it looks like the new IKVM can no longer support assemblies that need to expose classes to COM.

[wasabii]

So I just tried using regasm to generate a reg file (and also to register) IKVM.Java.dll directly, off of the develop branch, and it worked fine.

[wasabii]

C:\dev\ikvm\dist\bin\net461>regasm .\IKVM.Java.dll Microsoft .NET Framework Assembly Registration Utility version 4.8.4161.0 for Microsoft .NET Framework version 4.8.4161.0 Copyright (C) Microsoft Corporation. All rights reserved.

Types registered successfully

[wasabii]

It's interesting from your error that it sorta fails while specifically mentioning ComparatorGeneric: your type. If it was just blindly doing all of IKVM.Java, and failing, I'd expect it not to mention your type anywhere near the failure.

What's this ComparatorGeneric do?

[jhuber1965]

It worked for you becasue you did not create the tlb file with the /tlb option. The type library file (tlb) is needed for COM. If you use the tlb option, it will most likely fail.

[wasabii]

Okay. Yeah. TLB failed.

[wasabii]

So here's what I've found. You can use System.Runtime.InteropServices.TypeLibConverter to do the tlb export programatically. But this call is all internal to the CLR, and you can't modify it in anyway to avoid the error. So, no go there.

I created a quick sample application which uses IKVM.Java, but does not export any IKVM.Java types. The TLB for this generated successfully. This would be how I would recommend you proceed: do not export IKVM.Java. That means wrap your Java classes in a .NET class, and don't accept or return types from IKVM.Java.

Someday in the future we will be splitting IKVM.Java back up again. But it will be when we do JDK9 support. Since JDK9 actually did split up the Java modules in a defined way (java.base, java.util, javax.swing, etc). JDK8 has no defined method for splitting them, which is why they ended up combined in the new system. The .OpenJDK. DLLS didn't really respect any meaningful external convention, and simply made the new build system harder.

[jhuber1965]

Thank you for the suggestion.

This is just one of several assemblies. Our calculation engine has nearly two hundred classes. The beautiful thing about the current setup is that I can generate .NET DLLs that have COM support using ikvmc without having to touch Visual Studio, simply by annotating my Java code as follows (in addition to setting up an assembly.java file for the class)

@cli.System.Runtime.InteropServices.ClassInterfaceAttribute.Annotation(cli.System.Runtime.InteropServices.ClassInterfaceType.__Enum.AutoDual)
public class ShellCalcOrig {  
[...]
}

If I understand your suggestion, I would need to use Visual Studio to write wrappers that expose COM for each one of those classes. I certainly could do that, but it seems that there would be a significant amount of work involved, and the wrappers would add another layer of complexity and maintenance to the build process. Actually, these days, I rarely use Visual Studio.

At this point, I will probably just stick with the old IKVM, and look into the feasibility of the wrapper solution.

If you are eventually going to split IKVM Java back up anyway, could you split JDK8 along the lines of how JDK9 was split?

[wasabii]

This is just one of several assemblies. Our calculation engine has nearly two hundred classes.

I can see how this would be an issue.

If you are eventually going to split IKVM Java back up anyway, could you split JDK8 along the lines of how JDK9 was split?

It's technically possible. But a lot of manual work. The layout of the source of JDK8 is just a gigantic directory of class files, split by platform. Separate forks of some things for Windows/Linux/etc.

https://github.com/openjdk/jdk8/tree/master/jdk/src/share/classes

Where as the JDK9 source is actually laid out by Module:

https://github.com/openjdk/jdk9/tree/master/jdk/src

So yeah, we could build N different projects, taking the knowledge of which classes live in which JDK9 module, and picking those classes out of JDK8. One by one. And then making manual decisions for those classes which are new in JDK8. And we'd end up with a gigantic text file of each JDK8 class name, sorted by module, etc.

It's a lot of manual work and maintenance. And it has to be ongoing: OpenJDK does still do updates to JDK8 (for now). And we plan to adopt those updates before beginning work on JDK9. Those updates do add classes, some backports from JDK9, but some new things (TLS3, etc).

This is all why it was combined: it's combined in the JDK source. And why we planned to split it in JDK9: it's split in JDK9 source.

[jhuber1965]

Understood.

So, a bit off topic, one thing that I've not really understood about IKVM and these assemblies, and I've been using IKVM to create these assemblies since 2009...

I somehow figured out that I have to create and registger tlb files for IKVM.OpenJDK.Core.dll, IKVM.OpenJDK.Security.dll, and IKVM.OpenJDK.Util.dll in order to get the assemblies to work with COM. Since it has been 13 years ago, I don't remember if I read that in the old IKVM discussion forums, or if I figured it out by trial and error, or a combination. However, nearly all of the IKVM dlls are required for the assemblies to run. There is a screenshot below of the references in the utilities assembly. The IKVM dlls there reference other IKVM Dlls, so almost all of them are eventually referenced.

So, why isn't a tlb file required for every IKVM DLL that is ultimately referenced in my assembly, but only for the 3 IKVM DLLs I have listed above?

[wasabii]

From what I can see it has to deal with exported types. Types that leak from the assembly. The TLB has to declare these, and so needs the TLBs for their assemblies.

Public signatures. Base classes, etc. Anything that might be exposed to the caller.

If an assembly is simple used by another, but it's types don't leak, then it doesn't need to be exposed to COM.

public static SomeType Method(SomeOtherType foo)
{
   var i = new SomethingElse();
   i.DoThing();
  return new SomeType();
}

In the above, SomeType and SomeOtherType are exposed to any callers. The TLB will have to create a COM method for Method, accepting a SomeOtherType, and returning a SomeType. And so COM callers need to know what SomeOtherType and SomeType are. However: SomethingElse() isn't expose to anything. So it doesn't.

[jhuber1965]

Thanks. That makes sense.

I'm also not sure why regasm/tlbexp don't handle the issue with the com.sun.tools.doclint.Entity+__Enum. I have overloaded methods in my classes, and regasm/tlbexp deal just fine with those; i.e. rate(int x), rate(int x, int y) in Java become rate(x as Integer) and rate_2(x as Integer, y as Integer) in COM. In the Enum, com.sun.tools.doclint.Entity+__Enum.agrave and com.sun.tools.doclint.Entity+__Enum.Agrave have different numeric values, so it seems that tlbexp could decorate those Enum entries.

[wasabii]

I think it's just the case differences. Methods in COM are case-insensitive.

[jhuber1965]

Just did a little test...I added another method to a Java class with the same name but capitalized... Java code:

    public void getVersion(DataVersion dv1) {
        MiscUtils mu1 = new MiscUtils();
        mu1.getVersion(mu1, VERSION_BASE, dv1);
    }    

    public void GetVersion(DataVersion dv1) {
        MiscUtils mu1 = new MiscUtils();
        mu1.getVersion(mu1, VERSION_BASE, dv1);
    }    

regasm/tlbexp handled it just fine...

[wasabii]

Huh. Good point then. What the hecks.

[wasabii]

Try making two public fields that differ by case.

[jhuber1965]

No problems there either... Java

package com.stanref.st.utilities;

@cli.System.Runtime.InteropServices.ClassInterfaceAttribute.Annotation(cli.System.Runtime.InteropServices.ClassInterfaceType.__Enum.AutoDual)
public class UtilitiesMain {

    public double fTest1;
    public double FTest1;

    final public static String VERSION_BASE = "com/stanref/st/utilities/version";

[wasabii]

How about statics?

[jhuber1965]

No problems it seems. However, it appears that statics don't get exported (whether final or not). However, this is not surprising to me, because Enums don't get exported either; the Enum classes are empty in COM. Which makes regasm/tlbexp crashing on that Enum in IKVM.Java strange.

package com.stanref.st.utilities;

@cli.System.Runtime.InteropServices.ClassInterfaceAttribute.Annotation(cli.System.Runtime.InteropServices.ClassInterfaceType.__Enum.AutoDual)
public class UtilitiesMain {

    final public static double fTest1 = 1.0;
    final public static double FTest1 = 2.0;

    final public static String VERSION_BASE = "com/stanref/st/utilities/version";

public enum VarTubeSize {
        S_0_25(0.25), S_0_3125(0.3125), S_0_375(0.375), S_0_5(0.5), S_0_625(0.625), S_0_75(0.75), S_0_875(0.875), S_1_0(1.0), 
        S_1_125(1.125), S_1_25(1.25), S_1_375(1.375), S_1_625(1.625), S_X(9999.0);
         private final double od;

         private VarTubeSize(double c) {
             od = c;
         }
         public double getSize()
         {
             return od;
         }
    }