search

gary-rowe / hid4java

A cross-platform Java Native Access (JNA) wrapper for the libusb/hidapi library. Works out of the box on Windows/Mac/Linux.
MIT License
229 stars 71 forks source link

Add win32-aarch64 support #93

Closed tresf closed 5 months ago

tresf commented 4 years ago

@gary-rowe as ARM OSs are becoming more popular on Windows and inevitably macOS, can we help contribute binaries to you for those platforms? If so, how can we get started?

gary-rowe commented 4 years ago

Agreed. I'm hoping to get my cross-compilation environment to support ARM soon.

Just in case it's of interest, I have a Telegram group chat if you want to get involved.

gary-rowe commented 4 years ago

Quick update: I've released hid4java version 0.6.0 to Maven Central which has several improvements to JNA and hidapi.

In the examples code I've added an architecture detection logging message which should assist with selection of suitable ARM support. As I understand it, JNA supports linux-arm (32-bit hard float), linux-armel (32-bit EABI) and linux-aarch64 is directed at ARM v8 but I don't have any hardware that could trigger this detection.

Would it be possible for your team to run the 0.6.0 code on the ARM hardware and tell me what the platform output is? For example on OS X (10.15.5) the output is:

Platform architecture: x86-64
Resource prefix: darwin
tresf commented 4 years ago

Would it be possible for your team to run the 0.6.0 code on the ARM hardware and tell me what the platform output is? For example on OS X (10.15.5) the output is:

Windows

Fortunately, Microsoft recently released an alpha JVM for Windows on ARM. Unfortunately... unless you're aware of another way, think this will require JNA for these platforms is available first. Here's what happens when I try to echo the lines you've referenced. It does show win32-aarch64, so I think that helps?

public class Main {


    public static void main(String[] args) {

        System.out.println("Platform architecture: " + Platform.ARCH);
        System.out.println("Resource prefix: " + Platform.RESOURCE_PREFIX);

    }

}

Raw output from "java HidTest" ``` C:\Users\Owner\Desktop\HidTest\src>"C:\Program Files\OpenJDK\jdk-15-ea+25-windows-aarch64\bin\java.exe" -cp .;..\lib\* com.company.Main Exception in thread "main" org.hid4java.HidException: Hidapi did not initialise: Native library (com/sun/jna/win32-aarch64/jnidispatch.dll) not found in resource path (.;..\lib\hid4java-develop-SNAPSHOT.jar;..\lib\jna-5.5.0.jar) at org.hid4java.HidDeviceManager.(HidDeviceManager.java:87) at org.hid4java.HidServices.(HidServices.java:82) at org.hid4java.HidManager.getHidServices(HidManager.java:75) at org.hid4java.HidManager.getHidServices(HidManager.java:54) at com.company.Main.main(Main.java:10) Caused by: java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/win32-aarch64/jnidispatch.dll) not found in resource path (.;..\lib\hid4java-develop-SNAPSHOT.jar;..\lib\jna-5.5.0.jar) at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1032) at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988) at com.sun.jna.Native.(Native.java:195) at org.hid4java.jna.HidApiLibrary.(HidApiLibrary.java:39) at org.hid4java.jna.HidApi.(HidApi.java:60) at org.hid4java.HidDeviceManager.(HidDeviceManager.java:84) ... 4 more ```

Mac

I'm still trying to get my hands on an openjdk build for MacOS, even a Zero-Assembler build would be a nice start. I'll update as I find out more.

gary-rowe commented 4 years ago

Thanks for running the tests, Tres. That output clearly shows that JNA doesn't have a compiled DLL for the win32-aarch64 system. Still, that does back up my linux-aarch64 theory so that's progress. If I get some time I might have a crack at compiling a JNA DLL (if it's even possible to get hold of the source to it) and put together an experimental hid4java. I'll update here with any news on that front.

tresf commented 4 years ago

If I get some time I might have a crack at compiling a JNA DLL (if it's even possible to get hold of the source to it) and put together an experimental hid4java. I'll update here with any news on that front.

Ok, if you have arm64 MSVC++ installed I believe the main change is to the VsDevCmd.bat command:

- -arch=amd64
+ -arch=arm64 -host_arch=amd64

Took me a while for me to find this, wanted to share.

gary-rowe commented 4 years ago

Awaiting JNA update: JNA #1238 so that win32-aarch64 is supported.

matthiasblaesing commented 4 years ago

@gary-rowe https://github.com/java-native-access/jna/pull/1238 is about darwin / darwin-aarch64, not windows.

For anyone interested into getting JNA onto mac so aarch64: You need to get involved - I'll look into it, but quite frankly I'm the opposite of an apple fan and while I'm able to get access to mac os on x86-64, mac os aarch64 will not get near me. The current state is described in the above mentioned issue.

For support of windows on aarch64 the same is true. I use windows for work and don't own aarch64 hardware (at least none with desktop level performance). If noone steps up, it won't be done.

tresf commented 4 years ago

I have hardware for both and willing to help.

For support of windows on aarch64 the same is true. I use windows for work and don't own aarch64 hardware (at least none with desktop level performance). If noone steps up, it won't be done.

The current status is available on the JNA mailing list here: https://groups.google.com/g/jna-users/c/UxpFZP4g_ls/m/L3tors4tAQAJ

In short, libffi isn't compiling yet on my local machine and I'm still trying to work that out. The libffi project has a PR which adds CMake support which I'll try soon as I've had more success with CMake on Windows than I have using cygwin/msys2 + makefiles.

mac os aarch64 will not get near me.

I have hardware available via SSH for anyone willing to help. I haven't started testing https://github.com/java-native-access/jna/pull/1238 yet but I will make sure to offer any useful information there.

tresf commented 3 years ago

Just a status on the Windows-side of things, we're able to get libffi to compile targeting arm64/aarch64 using x86_64 Windows using the MSVC compiler. We've managed to get this to compile both using the cmake PR as well as master using MSVC tools, however chaining this into the JNA build script is proving difficult. Any pointers are welcome. Happy to share compile flags if needed. Pinging @Vzor-.

tresf commented 3 years ago

JNA for Windows ARM64 is available here https://github.com/java-native-access/jna/pull/1264. macOS to follow.

gary-rowe commented 3 years ago

I've created a branch issue-93 to cover the work for supporting win32-aarch64. My strategy is as follows:

  1. Rely on @tresf to build a local version of JNA using their build environment. The target version in the JNA pom.xml should be issue93-SNAPSHOT. (I've not attempted to build this myself)
  2. This should enable a local build of hid4java (versioned as develop-SNAPSHOT) that references jna-issue93-SNAPSHOT.
  3. This version can be tested on a traditional platform (e.g. Win32, Darwin) using the examples.

If the above goes well we can proceed to the next part of the plan.

  1. Rely on @tresf to build a local version of hidapi using an aarch64-win32 build environment. This is done as follows:
git checkout https://github.com/libusb/hidapi.git
cd hidapi
make clean
./bootstrap
./configure

The resulting DLL should appear in windows/.libs/libhidapi-0.dll.

  1. Rename the DLL to hidapi.dll and move it to a new directory <hid4java-root>/src/main/resources/win32-aarch64/hidapi.dll
  2. Build hid4java on branch issue-93 once again and deploy the JAR to a project running on an AArch64 machine.
  3. Check if the new DLL is picked up and used.

At this stage it's a shot in the dark but I'm hoping to get some insight into what needs to be done to support this.

If anyone can advise on a suitable cross compilation environment (ideally based on Linux) with GCC flags or Mingw32/64 that would speed things up a lot. See this excerpt from build-hidapi.sh for examples of the kind of settings I'm looking for (tricky parts are substituting for gcc-aarch64-linux-gnu and the like):

dockcross-windows-shared-x64 bash \
sudo dpkg --add-architecture arm64 && \
sudo apt-get update && \
sudo apt-get --yes install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libudev-dev:arm64 libusb-1.0-0-dev:arm64 && \
sudo make clean && \
sudo ./bootstrap && \
sudo ./configure --host=aarch64-w64-mingw64 CC=aarch64-w64-gnu-gcc && \
sudo make
tresf commented 3 years ago

If anyone can advise on a suitable cross compilation environment (ideally based on Linux) with GCC flags or Mingw32/64 that would speed things up a lot. See this excerpt from build-hidapi.sh for examples of the kind of settings I'm looking for (tricky parts are substituting for gcc-aarch64-linux-gnu and the like):

At the time of writing this, the only non-Windows win32-aarch64 compiler is Clang's. @matthiasblaesing showed doubt when mentioned, so we went the MSVC route through libffi's msvcc.sh wrapper. We may be able to enhance this with clang support in the upstream PR if warranted.

Assuming you're more concerned with the downstream hidapi compilation only, clang supports a -target <triple>. I'm not sure if there's similar mingw-... binary prefixes like gcc offers, or if the flags are expected. Unfortunately, clang offers no way to identify which options are available, this was also identified upstream in JNA (was specific to macOS, but same lack of arch/platform detection), however that's probably more commonly a versioning problem. Anyway, the source code suggests that the following triple would work:

-target aarch64-win32

This is untested.

tresf commented 3 years ago

FYI, split the macOS discussion over to https://github.com/gary-rowe/hid4java/issues/108 so we can focus on Windows support in one thread.

tresf commented 3 years ago

No luck with hidapi for aarch64-win32 on Clang (yet). It requires the Windows SDK which I don't have on Linux. I can probably trick it into using mingw's headers, but I'd rather find a tutorial for this, or use a wrapper.

Anyway, the .sln for hidapi builds just fine using Visual Studio 2019. I added the ARM64 Release build profile, then re-targeted the project to VS 2019 and it built a .dll.

tresf commented 3 years ago

This was done by:

@gary-rowe has expressed he builds his own binaries by hand, so at this point, I'm not adding anything to the issue-93 branch. 27ac2b6 is a stop-gap until JNA approves the upstream PR.

tresf commented 3 years ago

Agreed. I'm hoping to get my cross-compilation environment to support ARM soon.

Seems to work well thanks to the hard work of @mstorsjo:

  1. Snag a release of llvm-mingw
  2. Extract and add the bin to $PATH
  3. Pass --host=aarch64-w64-mingw32 to configure

Proof of concept gist here.

Edit: Shared with dockcross incase they find it useful.

tresf commented 3 years ago

FYI, aarch64 support has been added to JNA in a 5.7.0-SNAPSHOT for both Windows 64-bit and MacOS 64-bit here: https://groups.google.com/g/jna-users/c/UwUa5qWbIco/m/AMLX8FNlAQAJ.

tresf commented 3 years ago

Just an update, Windows ARM64 build support through dockcross has landed in the develop branch for those needing it. I'll be providing a snapshot release over on my fork as an interim until it's added officially.

gary-rowe commented 1 year ago

Good news! I've managed to compile a win32-aarch64 DLL using Dockcross on my OSX (darwin-aarch64) machine. This uses the latest JNA release (5.12.1) and hidapi source code. As a bonus for #108 there is darwin-aarch64 dylib in place as well so I'll need to do some cherry picking betwen branches or something.

@tresf Would you (or anyone, really) have some time to verify that the win32-aarch64 library works in principle (basic USB enumeration on real hardware)?

In anticipation of some success, I'll merge this branch into develop so I can press on with other issues.

tresf commented 1 year ago

@gary-rowe I'd be happy to test, I have two machines running ARM64, one physical and one VM. Do you have a binary to share, or are you asking about the binary that I created?

gary-rowe commented 1 year ago

Thanks @tresf . You can find the latest binaries in the develop branch. I’ll use that one for all fixes.

tresf commented 1 year ago

@gary-rowe I tested the produced hid4java-develop-SNAPSHOT.jar from the develop branch with Microsoft's OpenJDK 11 for ARM64 on a Surface Pro X running Windows 11 ARM64 and I was able to list, claim and release a device.

At time of writing this, I have not yet tested reading from a device (the device I use is currently in storage).

gary-rowe commented 1 year ago

That's great news, @tresf - thank you for looking at this. I'll continue adding code to the develop branch, but if you find any problems please report them here so we don't lose track.

gary-rowe commented 5 months ago

Support for linux-aarch64 is in develop so I'll close this for now.