RobLewis commented 3 years ago

[x] I have checked the documentation
[x] I have done a quick search in past issues or Stack Overflow for similar problems

I installed SDKMAN! on a Raspberry Pi 3B+ and it appears to work.

But I've tried installing 2 different versions of java and they both give the same error message when I try to check the installed version with java -version.

bash: /home/pi/.sdkman/candidates/java/current/bin/java: cannot execute binary file: Exec format error

This seems to indicate that the installed java was compiled for the wrong architecture, but I don't know what to do about it.

Raspbian OS has been freshly updated, with latest versions of everything.

RobLewis commented 3 years ago

I tried to revert to just doing a default Java install but now that doesn't work, apparently because SDKMAN has tampered with my PATH (with no warning). How can I uninstall it and get back to a default state? It's been nothing but trouble so far.

I found and looked at the uninstall instructions but they say nothing about fixing my PATH variable.

marc0der commented 3 years ago

Hi @RobLewis, I can confirm that we currently don't have arm binaries for this Java vendor. What you are seeing is a known issue in identifying the arm architecture. Would you please be able to provide us with the output of uname and uname -m on your Pi so we can try to fix this?

If you would like to uninstall sdkman follow the instructions on https://sdkman.io/install and open a new shell.

delight commented 3 years ago

Hi @marc0der I'm not Rob but on his behalf: On a Raspberry Pi 3B+ (with the official raspberry pi os - former raspbian) it is:

uname: Linux uname -m: armv7l

I saw that sdkman uses the following curl command for sdkman i 11.0.9.hs-adpt: curl --progress-bar --location -C - --retry-max-time 60 --retry 0 https://api.sdkman.io/2/broker/download/java/11.0.9.hs-adpt/linux64 --output /home/koko/.sdkman/tmp/java-11.0.9.hs-adpt.bin while it should be curl --progress-bar --location -C - --retry-max-time 60 --retry 0 https://api.sdkman.io/2/broker/download/java/11.0.9.hs-adpt/linuxARM --output /home/koko/.sdkman/tmp/java-11.0.9.hs-adpt.bin ^^^it is case insensitive, but for consistency with the previous code i used capital letters ARM as it was used for ARM64 (aarch64)

found that URL on api.sdkman.io by trial and error :see_no_evil:

At the end the version api (https://api.sdkman.io/2/candidates/java/linuxARM/versions/list?installed=) also needs to be adjusted - but that is out of my scope.

I'll send a pull request with the patch. For now (as long as the versions api is not adjusted) it prevents users from installing a version for a wrong architecture.

delight commented 3 years ago

https://github.com/delight/sdkman-cli/tree/feature/issue-836

842

marc0der commented 3 years ago

Hi @delight, unfortunately, this isn't as simple as it may seem. There are two approaches we can follow:

we could start supporting the architecture with actual binaries
we could block people from installing the wrong binary as you mentioned

We also require some additional work in the backend to accommodate this in either case.

If we did decide to go for this first approach, we need to host the appropriate binaries required to support it, which I don't think is the case right now. Are there such JDK binaries available? And if so, what vendors support them? Maybe @eddumelendez can also give some guidance here.

If the second approach is the only way out, we need to do some serverside validation to the architecture to block the users. This is done in the sdkman/sdkman-candidates service.

delight commented 3 years ago

Hi @marc0der,

well as I mentioned before the patch alone prevents the users already from installing wrong binaries on the raspi. My guess is that there are probably many people who ran into that problem before.

About the binaries, I know that Adoptopenjdk supports arm linux architecture. If I recall correctly they even have a swagger documented api to retrieve the binaries from their server.

I would suggest to merge the patch in the first step to avoid problems occurring for users and then when there is time/motivation populate the backend with the binaries.

In any case the current state is unsatisfactory for pi users.

Cheers, Kosta

marc0der commented 3 years ago

Hey Kosta,

Please don't misunderstand, but I would really like to address this in the right way, and very soon.

The matter is a little more complicated than simply passing an unknown value to the backend, as this does throw things off. Before the /download endpoint is hit on the broker service, other handshakes are performed such as the /validate endpoint on the candidates service. We want to accept know identifiers and handle them accordingly.

To add to the matter, the new Mac Apple Silicon laptops are also ARM-based, and we need to accommodate them too. All I'm proposing right now is to rework this code and fix this issue for all known platforms. I'm actually already looking at doing this work right now due to the urgency.

delight commented 3 years ago

Hi @marc0der,

I saw the code/rest requests that are executed before the download. I also tried the patch on a raspi3 and it simply delivers an empty list of versions for the candidates of java.

As for the arm based mac there is no conflict as the patch results in LinuxARM similar to LinuxARM64 for aarch64 architecture. So there is no expected conflict with OSX.

I also looked up other candidates with the patch applied on a pi3 as I did with direct calls to the rest api and things worked fine. E.g. listing the available groovy versions etc.

At the end the simple patch is for sure an improvement over the current situation because it prevents the current faulty behavior of installing wrong binaries - As we all want to avoid error-prone code.

I guess you need to start at some point and pick an identifier anyways. LinuxARM is consistent with the current implemention.

Ultimately it's up to you to decide the path of solving this issue. Looking forward to it.

Cheers, Kosta

marc0der commented 3 years ago

The architecture identifiers are now being propagated correctly to the backend. This has been released in the v5.9.2. It means that on Pi you will not see all the Java versions you saw before, those that are available should now probably be added via the database migrations. That or @eddumelendez could automate this somehow.

At least there won't be any more of this reported error. Can someone confirm that this works now?

marc0der commented 3 years ago

Not hearing anything back so I presume this is working now.

retrodaredevil commented 3 years ago

@marc0der I'm getting this same error on my Raspberry Pi Zero W (ARMv6 architecture). I wanted to get Java 11 working on it, which I know is difficult on ARMv6. I was able to get 11.0.10 zulu version working without sdkman (downloaded tar.gz manually).

Since I got it working manually without sdkman, I wanted to try sdkman to get it working, so I ran sdk install java 11.0.10-zulu. After that installs successfully, I get -bash: /home/pi/.sdkman/candidates/java/current/bin/java: cannot execute binary file: Exec format error after running java -version.

Here's some output:

$ sdk version

SDKMAN 5.11.0+644
$ uname -a
Linux raspberrypi 5.10.17+ #1403 Mon Feb 22 11:26:13 GMT 2021 armv6l GNU/Linux
$ uname -m
armv6l
$ cat ~/.sdkman/candidates/java/11.0.10-zulu/release
IMPLEMENTOR="Azul Systems, Inc."
IMPLEMENTOR_VERSION="Zulu11.45+27-CA"
JAVA_VERSION="11.0.10"
JAVA_VERSION_DATE="2021-01-19"u/release
... (some other stuff)

The version of the zulu jdk I got working manually is zulu11.45.27-ca-jdk11.0.10-linux_aarch32hf, which looks to me the same as the one sdkman installed.

I also got the same error with 11.0.10.j9-adpt, but I'm not sure if that's supposed to support ARMv6 like the zulu one is.

Also, the output of sdk list java is pretty big, I don't know if it's supposed to be that way (are all of these really supported on ARMv6).

Please let me know if you want more output, or if I'm mistaken and this is a completely different issue.

marc0der commented 3 years ago

@retrodaredevil I suspect we just have an incorrect version deployed for your architecture, perhaps @eddumelendez can advise as he curates the java releases.

retrodaredevil commented 3 years ago

@marc0der or @eddumelendez, I'm still having this error. Is this something that's on the radar to be fixed? Should I create a new issue for this? I've tried a few different versions from zulu and get the same error.

eddumelendez commented 3 years ago

Sorry for the the delay replying about this. I took a look a weeks ago about it and try to see which vendors are providing those binaries. TBH, it was a little confuse for me and since I don't own a raspberry pi, I am not able to play with it. It would be great if you can share the link for zulu so we can register it and start with this one on sdkman. The next thing to do is look for another vendors who provides binaries for raspberry.

There is a little difference in the binary you installed. It ends with hf which is not provided by sdkman. I just found the following link from Zulu's site. Currently only 8, 11 and 13 are provided. If those works for you we can add them if we already support armv6l.

@marc0der As far as I can see we don't support armv6l, correct me if I am wrong.

marc0der commented 3 years ago

@eddumelendez we currently support arm7l (32 bit) and arm8l (64 bit). We can certainly look at adding support for arm6l, although it would take some work on the backend and the CLI.

Also, we'll need to look at what the difference is between arm6l and arm7l in terms of compatibility, as these both seem to be 32-bit Linux architectures.

eddumelendez commented 3 years ago

I did some research those days and this is what I found

Azul Zulu provides binaries for armv6 here
BellSoft Liberica provides binaries for armv7 here are the links for 11 and 17

Sources:

pi4j installation
pi4j also points to another blog for armv7

This post is interesting because it says that the current openjdk linux aarch can be installed on raspberry pi

eddumelendez commented 3 years ago

I just found one of the @FDelporte blogposts and it's using sdkman.

@FDelporte could you help us providing more info about the device you used on this blogpost?

marc0der commented 3 years ago

Hi @FDelporte, would you be willing to help us with this if you are available?

retrodaredevil commented 3 years ago

I just found one of the @FDelporte blogposts and it's using sdkman.

sdkman works perfectly on Raspberry Pis as far as I know, it just doesn't work on Raspberry Pi Zeros (and likely Raspberry Pi 1s too), because it uses an armv6 architecture. This is why installation in that blog post worked fine without errors.

I have a few Raspberry Pis that I have access to (RPi0, RPi2, RPi3, RPi4 - all model Bs I think), so if you'd like to me to provide information on them or test different builds of Java I can. I'm only aware of sdkman not working on RPi0s, but I can test on the other models too.

marc0der commented 3 years ago

Thanks, @retrodaredevil! I think what we're mostly after is what uname returns for this arm6l architecture on RPi0. If you could provide us with the output of:

uname -a
uname -s
uname -m

it would already be super helpful for us to move ahead.

retrodaredevil commented 3 years ago

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.10.60+ #1449 Wed Aug 25 14:59:20 BST 2021 armv6l GNU/Linux
pi@raspberrypi:~ $ uname -s
Linux
pi@raspberrypi:~ $ uname -m
armv6l
pi@raspberrypi:~ $

retrodaredevil commented 3 years ago

I also forgot I have access to an RPi1 as well. I haven't attempted to use sdkman on it, but I assume it wouldn't work just like the RPi0.

joshua@doorpione:~ $ uname -a
Linux doorpione 5.10.17+ #1421 Thu May 27 13:58:02 BST 2021 armv6l GNU/Linux
joshua@doorpione:~ $ uname -s
Linux
joshua@doorpione:~ $ uname -m
armv6l
joshua@doorpione:~ $

Looks like that's basically the same output.

marc0der commented 3 years ago

Boom! Thank you! At least there are no surprises here, this is exactly what I was expecting.

It means that we will need to do some work in the CLI and backend to support these, @eddumelendez.

Also, we'll need to differentiate between this variant and armv7l, which is also a 32-bit architecture that we currently call LinuxARM32. Just me wondering but does the same Java binary run on both of these variants, or are these architectures distinct from each other?

retrodaredevil commented 3 years ago

So here's the output on an RPi2 of java -version and some uname outputs. The java I have installed there doesn't work on RPi1 or RPi0. The entire reason I want to get sdkman working on the RPi1 and RPi0 is that the builds for Java 11 in the official repositories for Raspberry Pi OS don't even work on RPi1s and RPi0s, so I know that armv7l and armv6l are different since the repositories don't even have an official build of Java 11 for armv6l.

If you'd like I can install sdkman on my RPi2, but hopefully this info is enough.

joshua@cabinpi:~ $ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-post-Raspbian-1deb10u1)
OpenJDK Server VM (build 11.0.11+9-post-Raspbian-1deb10u1, mixed mode)
joshua@cabinpi:~ $ uname -a
Linux cabinpi 5.10.17-v7+ #1421 SMP Thu May 27 13:59:01 BST 2021 armv7l GNU/Linux
joshua@cabinpi:~ $ uname -s
Linux
joshua@cabinpi:~ $ uname -m
armv7l
joshua@cabinpi:~ $

FDelporte commented 3 years ago

Seems I missed this whole discussion and questions, my bad!

For Pi Zero with ARMv6 indeed I only found Azul/Zulu to be working as I described on https://pi4j.com/documentation/java-installation/ And SDKMAN on Pi 4 with a 64bit system works perfectly as described on https://foojay.io/today/java-17-on-the-raspberry-pi/ to install e.g. Java 17.

How can I help to fix this issue? It would indeed be great to have SDKMAN working on all Pi versions, as that would simplify #JavaOnRaspberryPi articles a lot ;-)

Could the Disco API (https://github.com/foojayio/discoapi) by www.foojay.io and @hansolo be useful here?

FDelporte commented 3 years ago

I see Azul did a great thing on https://www.azul.com/downloads/?architecture=arm-32-bit-hf&package=jdk to differentiate between ARMv6 (ARM 32-bit SF) and ARMv7/8 (ARM 32-bit HF).

Screenshot 2021-09-30 at 08 12 03

FDelporte commented 3 years ago

If we want to support all Raspberry Pi versions with SDKMAN, I think these would be the next steps:

In the database with Java versions used by SDKMAN, it must be possible to link them to arm32-hf and arm32-sf
- Is this already possible?
- The Disco API by @hansolo shows that many arm32-versions are available, but the problem is that there is no standard to make the distinction between hf and sf.
- https://api.foojay.io/disco/v2.0/packages?architecture=arm&bitness=32
- So the challenge will be how to keep this up-to-date in the SDKMAN database
- Anything I can do here to assist?

Screenshot 2021-10-01 at 16 12 25 Screenshot 2021-10-01 at 16 15 43

in the CLI tool extra checks need to be integrated to detect the type of arm32
- The pull request https://github.com/sdkman/sdkman-cli/pull/842 by @delight already points in the right direction but needs further implementation to differentiate between ARMv6 and ARMv7/8
- I can assist with this task, or continue on top of the changes proposed by @delight

Who can decide or give feedback on these topics?

HanSolo commented 3 years ago

I could also offer an optional field named FPU to the Disco API which could have the values hardfloat, softfloat and unknown (default). Just let me know if this might be helpful...

marc0der commented 3 years ago

Our backend currently supports a LINUX_ARM32 platform, although we don't have any data in our database for this platform. From what I can tell, we would need to split LINUX_ARM32 into two, say, LINUX_ARM32HF and LINUX_ARM32SF.

@HanSolo, it would be awesome if you could surface the HF and SF variants for us in Disco.

@FDelporte, we have a daily job that scrapes Disco API for the latest JDK versions, which calls through to our Vendor API to publish any new versions. @eddumelendez maintains this job (and is also the curator of the JDKs on SDKMAN). He would need to make some changes in this job to accommodate HF/SF.

We would also need to support the new platform variants in our backend services before publishing these versions. I know my way around this, so I'm probably the best placed to do this work.

We would need to infer which variant we require for that particular arm-32 platform in the CLI. The inference is currently performed in the sdkman-init.sh using uname.

As always, contributions through PRs are always greatly appreciated!

HanSolo commented 3 years ago

I‘ve already implemented a version using a new parameter names „fpu“. It can have values like „hard_float“, „soft_float“ and „unknown“ (default). It‘s already working on my private server. Will ping Eddu as soon as it will be available in production.

FDelporte commented 3 years ago

@marc0der see commit in fork: https://github.com/FDelporte/sdkman-cli/commit/db38d2fd22987c2ea81d4379ef619254c26ca089

Is this OK for PR?

FDelporte commented 3 years ago

@marc0der modified based on your remarks and created https://github.com/sdkman/sdkman-cli/pull/991

FDelporte commented 3 years ago

@eddumelendez something I can contribute to support LINUX_ARM32HF and LINUX_ARM32SF for the daily Disco API job?

HanSolo commented 3 years ago

I think as soon as we have the current code in production Eddu could simply ask for the fpu field and store this info in the sdkman database. Then it will be up to sdkman how they present it to the users. Unfortunately it takes longer than I thought to bring this code to disco production but we are working on that.

FDelporte commented 3 years ago

Thanks @HanSolo! Small-step-by-step we change the world ;-)

marc0der commented 3 years ago

We still have some work on our own backend services to accommodate these variants @FDelporte.

FDelporte commented 2 years ago

@marc0der don't want to push, but anyhow doing it :-) ... is there a timing when this would be available?

marc0der commented 2 years ago

Apologies, things are a bit busy atm so not getting around to it. Hopefully I'll get some time over the Christmas break.

FDelporte commented 2 years ago

No problem :-) Let me know if you need some testing! Question: wouldn't it be an idea to show the detected platform in the sdk list output for instance?

marc0der commented 2 years ago

That's an excellent idea! Feel free to raise a feature request for that so I don't forget. It should be very simple to do.

marc0der commented 2 years ago

Hi all! :christmas_tree: :sun_with_face:

As promised, I'm spending some time implementing support for the ARM32 variants on the backend. Here are some PRs for the :eyes: of @eddumelendez and @helpermethod:

Broker service: https://github.com/sdkman/sdkman-broker/pull/12 Candidates service: https://github.com/sdkman/sdkman-candidates/pull/44 Vendor Release service: https://github.com/sdkman/vendor-release/pull/10 Database migrations: https://github.com/sdkman/sdkman-db-migrations/pull/568 Gradle release plugin: https://github.com/sdkman/sdkman-vendor-gradle-plugin/pull/15 Maven release plugin: https://github.com/sdkman/sdkman-vendor-maven-plugin/pull/6

This should get us into a position where we can publish our first ARM32 binaries. @eddumelendez, I suggest we first try a manual insert using the DB migrations before we automate this. WDYT?

Once we have our first version available, @FDelporte can kick the the tires to see if it is working.

eddumelendez commented 2 years ago

totally agree! 🚀

marc0der commented 2 years ago

I've now deployed all backend services :tada: @eddumelendez, would you be able to source some links for JDKs so that we can test this functionality?

eddumelendez commented 2 years ago

Zulu 8 and 11 has been released for SF and HF 🚀

I just noticed I missed a lot in the thread, sorry about that. I will be replying back next week. Hopefully with more updates.

retrodaredevil commented 2 years ago

First off, awesome work!

So it's showing the correct versions for Soft Float (which is the one that I was having the problem with initially), but I believe that there should be far more supported JDKs with Linux ARM 32bit Hard Float. (I think the versions shown there were correct before these updates)

I created another issue: #1030. It describes the problem of getting -bash: __sdkman_post_installation_hook: command not found after installation on probably all ARM 32Bit versions.

eddumelendez commented 2 years ago

@retrodaredevil I have added Liberica too! thanks for helping us testing

FDelporte commented 2 years ago

Wow this is really great!!!!

On a Raspberry Pi 4 with 32bit OS

$ uname -a
Linux crowpi 5.10.60-v7l+ #1449 SMP Wed Aug 25 15:00:44 BST 2021 armv7l GNU/Linux
$ uname -m
armv7l

This is result of sdk list java

================================================================================
Available Java Versions for Linux ARM 32bit Hard Float
================================================================================
 Vendor        | Use | Version      | Dist    | Status     | Identifier
--------------------------------------------------------------------------------
 Liberica      |     | 17.0.1       | librca  |            | 17.0.1-librca       
               |     | 11.0.13      | librca  |            | 11.0.13-librca      
 Zulu          |     | 11.0.13      | zulu    |            | 11.0.13-zulu        
               |     | 8.0.312      | zulu    |            | 8.0.312-zulu        
================================================================================

FDelporte commented 2 years ago

And on Raspberry Pi Zero 1 (new SD card with Raspberry Pi OS Lite)

$ uname -a
Linux zero-java-test 5.10.63+ #1459 Wed Oct 6 16:40:27 BST 2021 armv6l GNU/Linux
$ uname -m
armv6l

Result:

$ sudo apt install zip
$ curl -s "https://get.sdkman.io" | bash
$ source "/home/pi/.sdkman/bin/sdkman-init.sh"
$ sdk list java
================================================================================
Available Java Versions for Linux ARM 32bit Soft Float
================================================================================
 Vendor        | Use | Version      | Dist    | Status     | Identifier
--------------------------------------------------------------------------------
 Zulu          |     | 11.0.13      | zulu    |            | 11.0.13-zulu
               |     | 8.0.312      | zulu    |            | 8.0.312-zulu
================================================================================
$ sdk install java 11.0.13-zulu
-bash: __sdkman_post_installation_hook: command not found

Indeed, also results in bug soon to be fixed with https://github.com/sdkman/sdkman-hooks/pull/41

marc0der commented 2 years ago

I've just released @eddumelendez's fix so this should now be resolved.

FDelporte commented 2 years ago

That's fast! Installation succeeded but seems to be still the wrong Java runtime...

$ sdk install java 11.0.13-zulu

Downloading: java 11.0.13-zulu

In progress...

########################################################################################################################################################################################################## 100.0%

Repackaging Java 11.0.13-zulu...

Done repackaging...

Installing: java 11.0.13-zulu
Done installing!

Setting java 11.0.13-zulu as default.
pi@zero-java-test:~ $ java -version
java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory

sdkman / sdkman-cli

Support Raspberry Pi with Java on arm6l architecture #836

842