Unable to compile .jar files with Ikvm8.7 & .net-core7

[HackPoint]

When I compile the code with apache.calcite.core.jar

using org.apache.calcite.rel;
using org.apache.calcite.rel.core;
using org.apache.calcite.rex;

namespace CalciteIKVM;

public class Class1
{
    private RelTraitSet trait; // RelTraitSet isn't visible I have 
}

public class Class2 : Filter
{
    internal Class2(RelInput input) : base(input)
    {
    }

    // rts should be a RelTraitSet
    public override Filter copy(object rts, RelNode rn1, RexNode rn2)
    {
        return null;
    }

}

Attaching two example project one is with java that is allowing to use calcite.core objects and .net which doesn't

CalciteIKVM.zip CalciteTest.zip

Another way that I've tryed to use the library:

`

    <PackageReference Include="IKVM.Image" Version="8.7.0" />
    <PackageReference Include="IKVM.Image.JDK" Version="8.7.0" />
</ItemGroup>

<ItemGroup>
    <IkvmReference Include="../../tools/ikvm/jars/accessors-smart-2.4.7.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/aggdesigner-algorithm-6.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/antlr-runtime-3.5.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/apiguardian-api-1.1.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/asm-9.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/avatica-core-1.23.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/avatica-metrics-1.23.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/avatica-server-1.23.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/backport-util-concurrent-3.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-core-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-csv-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-elasticsearch-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-file-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-linq4j-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/calcite-server-1.34.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/checker-qual-3.12.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-codec-1.15.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-compiler-3.1.8.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-dbcp2-2.6.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-io-2.11.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-lang-2.6.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-lang3-3.8.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-logging-1.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-math3-3.6.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/commons-pool2-2.6.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/elasticsearch-rest-client-7.10.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/error_prone_annotations-2.11.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/failureaccess-1.0.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/guava-31.1-jre.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpasyncclient-4.1.4.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpclient-4.5.10.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpclient5-5.1.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpcore-4.4.12.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpcore-nio-4.4.12.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpcore5-5.1.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/httpcore5-h2-5.1.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/ical4j-1.0.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/j2objc-annotations-1.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jackson-annotations-2.14.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jackson-core-2.14.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jackson-databind-2.14.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jackson-dataformat-yaml-2.14.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/janino-3.1.8.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/javax.servlet-api-4.0.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jetty-http-9.4.44.v20210927.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jetty-io-9.4.44.v20210927.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jetty-security-9.4.44.v20210927.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jetty-server-9.4.44.v20210927.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jetty-util-9.4.44.v20210927.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/json-path-2.7.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/json-simple-1.1.1.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/json-smart-2.4.7.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jsoup-1.11.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jsr305-3.0.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jts-core-1.19.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/jts-io-common-1.19.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/memory-0.9.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/natty-0.13.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/opencsv-2.3.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/proj4j-1.2.2.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/protobuf-java-3.21.9.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/sketches-core-0.9.0.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/slf4j-api-1.7.33.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/snakeyaml-1.33.jar" />
    <IkvmReference Include="../../tools/ikvm/jars/uzaygezen-core-0.2.jar" />
</ItemGroup>`

This way if I compare to two assemblies

Old: New:

The only difference is old and new IKVM.net

For some reason the resources are compiled different too: Old: New:

Can you please check if it's possible that I made a mistake in configuration? Can you please assist?

[wasabii]

Use IKVM.Maven.SDK.

Otherwise, you will need to include, in IkvmReference.References, the full reference hierarchy, so that IKVM can build them in the proper order, depending on each other.

[HackPoint]

@wasabii I do use it, please check the CalciteIKVM.zip

   <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="IKVM.Maven.Sdk" Version="1.6.0" />
    </ItemGroup>

    <ItemGroup>
        <MavenReference Include="org.apache.calcite:calcite-core" Version="1.34.0" />
    </ItemGroup>`

[wasabii]

Wait.....

So everythign EXCEPT that one line works?

You got the wrong package name, dude.

RelTraitDef lives under org.apache.calcite.plan

[jeremyosterhoudt]

@wasabii the calcite.plan is part of the Calcite Core jar.

The issue we are seeing is RelTraitSet is not compiled into a class. places where RelTraitSet is taken as a parameter have been replaced with object. We see this with a couple of other classes as well, but RelTraitSet is the easiest one to show in an example.

Are there any knobs or logging levels that can be enabled to track down why this class is not being generated when we try to pull it into .NET?

[wasabii]

I added a using statement to your sample project, referencing the right namespace, and it worked.

[jeremyosterhoudt]

That's interesting, for me defining the namespace or using the fully qualified class name (org.apache.calcite.plan.RelTraitSet) doesn't compile.

I'm usingIKVM.Maven.Sdk 1.6.0 on an Apple Silicon based Mac. Could that be the issue? Which version were you able to compile with out of curiosity?

This is wha I see in my local project for reference.

[wasabii]

Wait a sec. The file I received from you had RelTraitRef used in Class1. And that was in the plan namespace.

Changed it to Set, and it now fails. I see what's going on.

During compilation, a number of errors are emitted:

ERROR: warning IKVMC0101: Unable to compile class "org.apache.calcite.plan.RelTraitSet"
ERROR:     (class format error "Unspecified class file format error")

This is an error parsing the RelTraitSet.class file. And I've traced it down to an actual bug. I'm pretty sure this bug isn't new to 8.7.0. Where it should have read a single byte, it was reading two bytes. As defined by the spec: https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.20; the size of type_argument_index is 1 byte. However, the IKVM.ByteCode parsing code was attempting to read two bytes (a short).

This will require a hotfix. Which I'll get out in a couple hours.

[jeremyosterhoudt]

Apologizes for the confusion! Thank you so much for looking into this and for the quick response!!

[HackPoint]

@wasabii which version in nuget should be with the hotfix?

[wasabii]

@HackPoint

https://github.com/ikvmnet/ikvm/pkgs/nuget/IKVM/142804693

In GitHub nuget repo. I haven't run it through any manual tests yet.

[wasabii]

Confirmed, it does solve the issue with RelTraitSet.

[HackPoint]

Thank you so much you are awesome!

Just bring to your knowledge: After you full release this package I believe it would be fixed. 0>Microsoft.NET.RuntimeIdentifierInference.targets(314,5): Message NETSDK1057 : You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy 0>CalciteIKVM.csproj: Warning NU1603 : CalciteIKVM depends on IKVM (>= 8.7.1-pre.3) but IKVM 8.7.1-pre.3 was not found. An approximate best match of IKVM 8.7.1 was resolved. CalciteIKVM -> CalciteIKVM/bin/Debug/net6.0/CalciteIKVM.dll

[HackPoint]

@wasabii I'm sorry I am disturbing you

After adding this:

  <ItemGroup>
        <PackageReference Include="IKVM" Version="8.7.1-pre.3" />
        <PackageReference Include="IKVM.Maven.Sdk" Version="1.6.1" />
    </ItemGroup>

    <ItemGroup>
        <MavenReference Include="org.apache.calcite:calcite-core" Version="1.34.0" />
        <MavenReference Include="org.apache.calcite.avatica:avatica-server" Version="1.23.0" />
        <MavenReference Include="org.apache.calcite:calcite-csv" Version="1.34.0" /> <--- this is the refence that breaks
    </ItemGroup>

Error: 0>IKVM.Maven.Sdk.targets(96,9): Error : DependencyResolutionException: Could not find artifact javax.jms:jms:jar:1.1 in central (https://repo1.maven.org/maven2/), try downloading from http://java.sun.com/products/jms/docs.html at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(RepositorySystemSession session, DependencyRequest request) at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.ResolveCompileDependencyGraph(IkvmMavenEnvironment maven, RepositorySystemSession session, IList1 repositories, IList1 items) in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 254 at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.ResolveReferences(IList1 repositories, IList1 items) in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 193 at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.Execute() in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 151 ArtifactResolutionException: Could not find artifact javax.jms:jms:jar:1.1 in central (https://repo1.maven.org/maven2/), try downloading from http://java.sun.com/products/jms/docs.html at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(RepositorySystemSession session, Collection requests) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(RepositorySystemSession session, DependencyRequest request) ArtifactNotFoundException: Could not find artifact javax.jms:jms:jar:1.1 in central (https://repo1.maven.org/maven2/), try downloading from http://java.sun.com/products/jms/docs.html As a result all dependencies are being deleted ( aka not created )

[wasabii]

Definately a new issue. Moving to ikvm-maven.