Closed BiologyTools closed 8 months ago
wasabii
commented
8 months ago Which previous version, curiously?
What's the full command line?
The fact that you're merging directories and stuff is odd. Tools don't need the libraries, which is why they're not packaged together. The tools need to point to the libraries using -reference.
But also, again, this whole process is best done through the MSBuild support.
BiologyTools
commented
8 months ago I've done it that way since I started using IKVM so since 8.3.3. I've also turned the created dll into a nuget package BioFormats.NET6. The command line is ./ikvmc bioformats_package.jar -target:library -recurse:dependencies *> output.txt The dependencies contains just a single jar file google-collect-1.0.jar What do you mean better done through MSBuild exactly? Do you mean I should do something like this, in my project file?
<IkvmReference Include="google_collect-1.0.jar">
<AssemblyName>google_collect</AssemblyName>
<AssemblyVersion>1.0.0.0</AssemblyVersion>
</IkvmReference>
<IkvmReference Include="bioformats_package.jar">
<AssemblyName>bioformats_package</AssemblyName>
<AssemblyVersion>7.0.1.0</AssemblyVersion>
<References>google_collect-1.0.jar</References>
<Aliases>bioformats_package</Aliases>
</IkvmReference>I would rather do it with ikvmc.exe since I've made the nuget package with the dll the same way since the beginning.
wasabii
commented
8 months ago Yeahhhhhh....
Okay. So, first, this is some advice on how packaging in .NET works. You should have a single NuGet package with both .NET Framework, and .NET 6.0 packages in it. That's how it's supposed to be set up. So you have a lib/net6.0 directory and a lib/net472 directory inside the package, with a different DLL in each, but named the same. Else, people can't add your package as a reference to a multitargeting project.
For building .NET Core (3.1,5,6) projects, you need to use the .NET 6 version of ikvmc. And to use this version, you need to specify either -lib pointing to the .NET SDK you are linking against, or -reference specifying every required DLL from the .NET 6 SDK (Located in Program Files\dotnet\etc). You need to specify the DLLs by hand, since there are multiple versions of .NET you could be linking against, and the tool cannot auto discover that information. If you do not use the .NET 6 version of ikvmc, you will be generating a .NET Framework DLL: that is, the DLL that is output from the .NET 472 version of the tool will have references to mscorlib (which no longer exists as a real library on Core, and is instead just a shim assembly). And this is the case. If you look in your BioFormats.NET6 package, you'll find content/bioformats_package.dll. If you open this DLL in a disassembler, you'll see it's NOT marked as a .NET Core assembly, instead, it has a reference to mscorlib from .NET Framework 4.8.1 on it.
And since you're putting the assembly into content and contentFiles, you have two copies of it distributed. And, since it's in contentFiles, it's not treated as a reference, it's treated as content, to be output on build. If you add your package to a project in VS, Intellisense does not work, and it does not compile, since nothing in contentFiles is treated as a reference. To be a reference it must be in lib.
And even then, since its a contentFile it DOES get copied to bin/ on build. However, since it's a contentFile, it doesnt' get added to .deps.json by a consuming project, and is thus not locatable at runtime, except by relative path. There is also a lib you did include named BioFormats.NET6, but which has no classes nor references in it, so it seems unused? This is a valid .NET6 library.
So... the error you posted about.... is this about trying to get the .NET Framework version of the tools to emit a .NET 6 library? Or is it about using the .NET 6 tools to do so? If it's about the .NET 6 tools, then the problem is you need to specify the entire target .NET SDK with either -reference or -lib. If it's about the Framework version trying to generate a .NET 6 library: it won't and can't.
There's a reason I strongly recommend using the built in functionality we have for working with Java libraries.
wasabii
commented
8 months ago Even if you do get this to work, since you're not following the guidance on IKVM's page, you're going to also be stomping on users that use your package in conjunction with other IKVM libraries that rely on google-collect. Since you're distributing your own embedded version of those same classes, if users add a reference to your library, and also some other library that uses google-collect, there will be two copies floating around, and likely the compiler will break with "unable to cast someobect to someobject". Same class name, but different class, etc.
BiologyTools
commented
8 months ago Well I've been wanting to make it into a multi targeted package but there is already a package called BioFormats.Net which is no longer being updated and I followed his notes on how he created the package to update it. https://github.com/GDanovski/BioFormats.Net This is how I stumbled upon IKVM and ikvmc.exe in the first place. I'm a more of a biologist and a self taught programmer so sorry if my questions may seem dumb. But I tried to use the NET6 tools to create the dll but I get the same error message as before. Well considering everything the way I've made the dll & package it has worked suprisingly well for all my .NET6 projects including on Linux & Mac x64.
I did also try the net6 version of the tools but I got the same error as with the net472 tools. When running ikvmc.exe:
./ikvmc : IKVM.Tools.Importer (8.7.0+Branch.tags-8.7.0.Sha.8fd37569453a3889205c22f1547e0609545cab4a)
At line:1 char:1
+ ./ikvmc bioformats_package.jar -target:library -recurse:dependencies ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (IKVM.Tools.Impo...7e0609545cab4a):String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Copyright c 2023 Jeroen Frijters, Windward Studios, Jerome Haltom, Shad Storhaug
Error: core library not found
*** INTERNAL COMPILER ERROR ***
PLEASE FILE A BUG REPORT FOR IKVM.NET WHEN YOU SEE THIS MESSAGE
IKVM.Tools.Importer, Version=8.7.0.0, Culture=neutral, PublicKeyToken=13235d27fcbfff58
F:\ikvm\8.7.0net6\ikvmc\net6.0\win-x64\
6.0.24 64-bit
System.IO.FileNotFoundException: System.Runtime
at IKVM.Reflection.Universe.Load(String refname, Module requestingModule, Boolean throwOnError) in
/_/src/IKVM.Reflection/Universe.cs:line 564
at IKVM.Tools.Importer.AssemblyResolver.Init(Universe universe, Boolean nostdlib, IList`1 references, IList`1
userLibPaths) in /_/src/IKVM.Tools.Importer/AssemblyResolver.cs:line 84
at IKVM.Tools.Importer.IkvmImporterInternal.Compile(String[] args) in
/_/src/IKVM.Tools.Importer/IkvmImporterInternal.cs:line 181
at IKVM.Tools.Importer.IkvmImporterInternal.Execute(String[] args) in
/_/src/IKVM.Tools.Importer/IkvmImporterInternal.cs:line 111Then I've tried setting up the IKVM reference in the project file as the readme suggest. But then visual studio still gives me errors like CS0246 The type or namespace name 'ome' could not be found (are you missing a using directive or an assembly reference?)
wasabii
commented
8 months ago Well, this is why I recommend the Maven approach. It is incredibly easy, as you just make a project and add a MavenReference, and it fully supports NuGet publish, which auto generates proper NuGet packages for you.
If you can't or don't want to do that, you're going to have to tackle the complexity inherent in this. ikvmc is no longer a single tool that magically discovers the .NET Framework. Since .NET Core, there are multiple version you can have installed at the same time, with multiple versions of the same assembly for each (reference, impl, shim) and YOU have to choose which one to use.
If I look in my C:\Program Files\dotnet\sdk directory right now, I have 5 copies of .NET 6 installed, at different versions. 6.0.203, 6.0.301, 6.0.etc, etc. These correlate with C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref, and other runtime packs, containing the .NET 6 reference assemblies.
And to do that, you need to pass -reference:C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.V\ref\net6.0\System.dll to ikvmc for every runtime assembly that is required. That's like 40 items. I am quite seriously telling you that your command line is going to need to consist of like ~40 -reference arguments, pointing to each .NET 6 assembly at the version you want to link to, as well as pointing to the specific IKVM.Runtime.dll and IKVM.Java.dll you want to link against. And there is no longer just a single mscorlib.dll that contains everything: the framework has been split across over 250 assemblies.
This is what has changed in .NET Core: there is no longer one installed system-wide copy of Framework. So the tool cannot "just use that". The user has to specify at compile time which assemblies he wants to link to.
This is the same for any other .NET compilation tool, these days. For instance, you can no longer compile simple C# by just running C:\Windows\Microsoft.NET\Framework\v4.0\csc.exe, and have it magically find mscorlib, System, System.Collections, System.Core, etc. The new csc.exe needs to have a -reference path to each assembly you require. IKVM is no different in this regard.
BiologyTools
commented
8 months ago I tried using the maven approach but it can't seem to find the bioformats_package on maven. I get the following error in VS:
DependencyResolutionException: Could not find artifact ome:pom-bio-formats:jar:7.0.0 in central (https://repo1.maven.org/maven2/)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(RepositorySystemSession session, DependencyRequest request)
at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.ResolveCompileDependencyGraph(IkvmMavenEnvironment maven, RepositorySystemSession session, IList`1 repositories, IList`1 items) in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 254
at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.ResolveReferences(IList`1 repositories, IList`1 items) in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 193
at IKVM.Maven.Sdk.Tasks.MavenReferenceItemResolve.Execute() in C:\w\ikvm-maven\src\IKVM.Maven.Sdk.Tasks\MavenReferenceItemResolve.cs:line 151
ArtifactResolutionException: Could not find artifact ome:pom-bio-formats:jar:7.0.0 in central (https://repo1.maven.org/maven2/)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(RepositorySystemSession session, Collection requests)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(RepositorySystemSession session, DependencyRequest request)
ArtifactNotFoundException: Could not find artifact ome:pom-bio-formats:jar:7.0.0 in central (https://repo1.maven.org/maven2/)Can I specify which repository it looks for the package in? Since bioformats should be on maven. https://mvnrepository.com/artifact/ome/pom-bio-formats/7.0.0
wasabii
commented
8 months ago I think we talked about this awhile ago, and the problem was that those libraries were not in Maven Central. They are in some third party repository. To add a reference to them, you'll need to enable that third party repository for your project.
That POM file from 'scijava', indicates the primary repository is "ome" at https://artifacts.openmicroscopy.org/artifactory/maven/
To add that to your project, set a MSBuild property:
<PropertyGroup>
<MavenAdditionalRepositories>$(MavenAdditionalRepositories);ome=https://artifacts.openmicroscopy.org/artifactory/maven/</MavenAdditionalRepositories>
</PropertyGroup>To explain some context: just like NuGet, there isn't necessarily one global repository. People can make their own. Just like, for instance, nuget.org tends to be the 'official run by Microsoft one', Maven Central tends to be the 'official public one, though not run by anybody in particular'.
The web site you mentioned, mvnrepository.com, isn't related to Maven Central. It's just some random website out on the Internet which catalogs other Maven repositories.
BiologyTools
commented
8 months ago Thank you for the explanation. I will read more and try to get this to work, then post the answer here once I do figure it out. Thank you Jerome for all the help so far. I'm new to java so .pom files etc. are all new to me.
BiologyTools
commented
8 months ago I have now managed to get the bioformats_package and another artifact IKVM complained about edu.ucar cdm-core added to the project. But now I'm getting the following error:
IKVMSDK0004 The IkvmReference maven$org.json:json:20230227 has invalid AssemblyName 'org.json' or AssemblyVersion '20230227.0.0.0'.Is there a way to define the assembly and assembly version of the MavenReference to fix the invalid name or version issue?
wasabii
commented
8 months ago Ahh. Yes. But it's a hack. What we've got here is a legit bug.
https://github.com/ikvmnet/ikvm-maven/issues/19
For now, can you follow the workaround mentioned on that page? For now. This needs to be fixed.
BiologyTools
commented
8 months ago That worked thank you so much no errors anymore! Now I just need to create a nuget package from this. :)
wasabii
commented
8 months ago Do you, though?
BiologyTools
commented
8 months ago Well at least I need to document this somewhere in my BioFormats.NET6 repository's readme. Plus I imagine I will have to add references and fix possible issues that could pop up each time a new bioformats_package is released. So far with almost every release something has changed and I've had to add artifacts. Plus I need to tell the OME team about how this is done since they have been hoping to automate this process, especially now that IKVM supports Arm64 since surprisingly many scientists use apple silicon. Though I haven't found many people developing with bioformats with C#. But hopefully this will get more people on board.
wasabii
commented
8 months ago Yeah.
The one thing though is, unless you're adding C# code along with their package, such as extension methods or the like: there's no real reason for you to distribute a NuGet package at all. Anybody can add the Maven reference you added directly to their project.
BiologyTools
commented
8 months ago Hmm that's true they could probably just add instructions on their website regarding how to use bioformats on C# rather than a nuget package. But I still have an issue, I got the project to build but when I try to run I get the following error:
System.TypeInitializationException
HResult=0x80131534
Message=The type initializer for 'IKVM.Runtime.LibJvm' threw an exception.
Source=IKVM.Runtime
StackTrace:
at IKVM.Runtime.LibJava..ctor()
This exception was originally thrown at this call stack:
IKVM.Runtime.LibJvm.LibJvm()
IKVM.Runtime.LibJvm.LibJvm()
Inner Exception 1:
InternalException: Could not load libjvm.
Any idea what could be causing this?
wasabii
commented
8 months ago In your build output, is there an ikvm/ directory?
Within that directory should be a osx-arm64/ directory, and within that bin/, and within that, libjvm.dylib?
BiologyTools
commented
8 months ago It runs perfectly now! I just had to delete the ikvm folder and rebuild. Thank you again for everything. On the OME forum I said I would have this done today and now it works thanks to you :)
wasabii
commented
8 months ago No problem! Good luck.
wasabii
commented
8 months ago @BiologyTools try 8.7.1 with the msbuild hack removed
I'm trying to convert a jar file to dll which has so far always worked with previous versions of IKVM but now with 8.7.0 I get "Error: core library not found" what could be the reason for it?
Here is my workflow incase I have done something wrong it has always worked in the past. I downloaded both IKVM-8.7.0-bin-net472.zip and IKVM-8.7.0-tools-net472-win-x64.zip from releases then extracted them into the same folder along with the bioformats_package_jar and it's dependencies placed placed in a folder "dependencies". Then ran ikvmc.
Here is the output I get from ikvmc: