Bug: Sap Machine OpenJDK fails to install

norrisjeremy commented 3 years ago

Bug report

None of the Sap Machine OpenJDK distros appear to be installable.

To reproduce

%sdk i java 16-sapmchn
Downloading: java 16-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 16-sapmchn at this time...

%sdk i java 15.0.2-sapmchn
Downloading: java 15.0.2-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 15.0.2-sapmchn at this time...

%sdk i java 11.0.10-sapmchn
Downloading: java 11.0.10-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 11.0.10-sapmchn at this time...

System info

%uname -rsv
Darwin 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64

%echo $SHELL
/bin/zsh

%/bin/zsh --version
zsh 5.8 (x86_64-apple-darwin20.0)

%sdk version
SDKMAN 5.11.0+644

%sha256sum .sdkman/tmp/java-16-sapmchn.bin
d0d073685f15ed76eba35b54d0097ab11c9252aad59d2fc9fa52d347440bfc4a  .sdkman/tmp/java-16-sapmchn.bin

%wget -q https://github.com/SAP/SapMachine/releases/download/sapmachine-16/sapmachine-jdk-16_osx-x64_bin.dmg && sha256sum sapmachine-jdk-16_osx-x64_bin.dmg
d0d073685f15ed76eba35b54d0097ab11c9252aad59d2fc9fa52d347440bfc4a  sapmachine-jdk-16_osx-x64_bin.dmg

%sha256sum .sdkman/tmp/java-15.0.2-sapmchn.bin
35fd651595cac2588de08a943e8bb61ee2dd10146f4b459511ae577e9c3fa42c  .sdkman/tmp/java-15.0.2-sapmchn.bin

%wget -q https://github.com/SAP/SapMachine/releases/download/sapmachine-15.0.2/sapmachine-jdk-15.0.2_osx-x64_bin.dmg && sha256sum sapmachine-jdk-15.0.2_osx-x64_bin.dmg
35fd651595cac2588de08a943e8bb61ee2dd10146f4b459511ae577e9c3fa42c  sapmachine-jdk-15.0.2_osx-x64_bin.dmg

%sha256sum .sdkman/tmp/java-11.0.10-sapmchn.bin 
9534d6ec89f1dd77615f20f0d3bff7dd4d3f76ba2a4fe2e29431485cd9131dbe  .sdkman/tmp/java-11.0.10-sapmchn.bin

%wget -q https://github.com/SAP/SapMachine/releases/download/sapmachine-11.0.10/sapmachine-jdk-11.0.10_osx-x64_bin.dmg && sha256sum sapmachine-jdk-11.0.10_osx-x64_bin.dmg
9534d6ec89f1dd77615f20f0d3bff7dd4d3f76ba2a4fe2e29431485cd9131dbe  sapmachine-jdk-11.0.10_osx-x64_bin.dmg

marc0der commented 3 years ago

Thanks for reporting this, @norrisjeremy. Perhaps @eddumelendez can advise on this as he is the curator of Java on SDKMAN.

eddumelendez commented 3 years ago

thanks @norrisjeremy for the report! last week I noticed it but couldn't fix it. I will look for some time to fix it this week. This is only happening for macos.

marc0der commented 3 years ago

Could it be due to an irregular internal structure of the binaries?

norrisjeremy commented 3 years ago

Could it be due to an irregular internal structure of the binaries?

I think the problem is that they are .dmg files, but SDKMAN attempts to treat them as .tar.gz files:

%cat .sdkman/tmp/hook_post_java_16-sapmchn.sh
#!/bin/bash
#Post Hook: osx-java-tarball
function __sdkman_post_installation_hook {
    __sdkman_echo_debug "A Mac OSX post-install hook was found for Java 16-sapmchn-openjdk."

     __sdkman_validate_binary_input "$binary_input" || return 1

    local present_dir="$(pwd)"
    local work_dir="${SDKMAN_DIR}/tmp/out"
    local work_jdk_dir="${SDKMAN_DIR}/tmp/Java-16-sapmchn"

    echo ""
    __sdkman_echo_green "Repackaging Java 16-sapmchn..."

    mkdir -p "$work_dir"
    /usr/bin/env tar zxf "$binary_input" -C "$work_dir"

    cd "$work_dir"/*/Contents
    mv Home "$work_jdk_dir"
    cd "${SDKMAN_DIR}"/tmp
    /usr/bin/env zip -qyr "$zip_output" "Java-16-sapmchn"
    cd "$present_dir"

    echo ""
    __sdkman_echo_green "Done repackaging..."

    __sdkman_echo_green "Cleaning up residual files..."
    rm "$binary_input"
    rm -rf "$work_dir"
    rm -rf "$work_jdk_dir"
}

function __sdkman_validate_binary_input {
    if ! tar tzf "$1" &> /dev/null; then
        echo "Download has failed, aborting!"
        echo ""
        echo "Can not install java 16-sapmchn at this time..."
        return 1
    fi
}

marc0der commented 3 years ago

I see. If they choose to force such packaging on their users we won't be able to support them any longer on sdkman.

marc0der commented 3 years ago

WDYT, shall we pull the osx versions @eddumelendez?

norrisjeremy commented 3 years ago

Have you considered contacting them? They openly advertise SDKMAN as an installation method. See https://github.com/SAP/SapMachine/wiki/Installation#-sdkman

norrisjeremy commented 3 years ago

FYI, they do appear to still offer .tar.gz downloads:

https://github.com/SAP/SapMachine/releases/download/sapmachine-16/sapmachine-jdk-16_osx-x64_bin.tar.gz

https://github.com/SAP/SapMachine/releases/download/sapmachine-15.0.2/sapmachine-jdk-15.0.2_osx-x64_bin.tar.gz

https://github.com/SAP/SapMachine/releases/download/sapmachine-11.0.10/sapmachine-jdk-11.0.10_osx-x64_bin.tar.gz

The problem seems to be that SDKMAN is pulling the .dmg files, instead of the .tar.gz files.

eddumelendez commented 3 years ago

currently, I am using this https://sap.github.io/SapMachine/assets/data/sapmachine_releases.json to get the bin but looks like it is returning just the dmg. However the release contains all binaries https://github.com/SAP/SapMachine/releases/tag/sapmachine-16 . I have also check foojay api and tar.gz is returned. Foojay integration is almost done so I think in the future we will not have those issues.

@marc0der I will send you the script to fix sapmachine osx binaries

marc0der commented 3 years ago

@eddumelendez if we can get the patch functionality reviewed and approved you could do it through the API instead. You could be the first to try it out, wdyt?

eddumelendez commented 3 years ago

urls for sapmachine were fixed.

norrisjeremy commented 2 years ago

FYI, the new 11.0.12 OpenJDK release from Sap Machine is failing to install again, in the exact same manner with earlier releases:

%sdk i java 11.0.12-sapmchn

Downloading: java 11.0.12-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 11.0.12-sapmchn at this time...

Examining the file downloaded in the .sdkman/tmp/java-11.0.12-sapmchn.bin directory reveals that it is once again downloading a .dmg image instead of a .tar.gz.

eddumelendez commented 2 years ago

sadly, we are taking as a source of truth this file which provides dmg. I will fix this later today to add the tar.gz Add fix the root cause hopefully at the end of this week.

eddumelendez commented 2 years ago

now the right binary should be installed.

norrisjeremy commented 2 years ago

FYI, the new 16.0.2 OpenJDK release from Sap Machine is failing to install, in the exact same manner as previously noted with earlier releases:

%sdk i java 16.0.2-sapmchn

Downloading: java 16.0.2-sapmchn

In progress...

######################################################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 16.0.2-sapmchn at this time...

Examining the file downloaded in the .sdkman/tmp/java-16.0.2-sapmchn.bin directory reveals that it is once again downloading a .dmg image instead of a .tar.gz.

eddumelendez commented 2 years ago

url updated.

norrisjeremy commented 2 years ago

FYI, the new 17.0 OpenJDK release from Sap Machine is failing to install, in the exact same manner as previously noted with earlier releases:

%sdk i java 17-sapmchn

Downloading: java 17-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 17-sapmchn at this time...

Examining the file downloaded in the .sdkman/tmp/java-17-sapmchn.bin directory reveals that it is once again downloading a .dmg image instead of a .tar.gz.

eddumelendez commented 2 years ago

fixed! sorry about still having this issue with sapmachine.

norrisjeremy commented 2 years ago

No problem, thanks! :)

norrisjeremy commented 2 years ago

FYI, the new 11.0.13 OpenJDK release from Sap Machine is failing to install, in the exact same manner as previously noted with earlier releases:

%sdk i java 11.0.13-sapmchn

Downloading: java 11.0.13-sapmchn

In progress...

######################################################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 11.0.13-sapmchn at this time...

Examining the file downloaded in the .sdkman/tmp/java-11.0.13-sapmchn.bin directory reveals that it is once again downloading a .dmg image instead of a .tar.gz.

norrisjeremy commented 2 years ago

FYI, the new 17.0.1 OpenJDK release from Sap Machine is failing to install, in the exact same manner as previously noted with earlier releases:

%sdk i java 17.0.1-sapmchn

Downloading: java 17.0.1-sapmchn

In progress...

####################################################################################################################################################################################################################### 100.0%
Download has failed, aborting!

Can not install java 17.0.1-sapmchn at this time...

Examining the file downloaded in the .sdkman/tmp/java-17.0.1-sapmchn.bin directory reveals that it is once again downloading a .dmg image instead of a .tar.gz.

marc0der commented 2 years ago

@eddumelendez any ideas?

marc0der commented 2 years ago

@norrisjeremy I've removed the offending binary from the database for now. Let's hear what @eddumelendez suggests we do to set this right permanently. He's the master of JDKs on SDKMAN! :smile:

eddumelendez commented 2 years ago

I have fixed the values. I haven't had much time to work on the fix for this but hopefully for next release.

norrisjeremy commented 2 years ago

This is happening yet again with the new 17.0.2 & 11.0.14 releases. Is there a plan to ever fix this permanently, or will I always be required to respond to this issue to trigger somebody to manually correct the problem?

montoyaedu commented 2 years ago

Hello,

It is happening also with 11.0.12-open

Downloading: java 11.0.12-open

In progress...

######################################################################## 100.0%
Download has failed, aborting!

Can not install java 11.0.12-open at this time...

eddumelendez commented 2 years ago

@montoyaedu we don't have any record for 11.0.12-open for MACOS.

will I always be required to respond to this issue to trigger somebody to manually correct the problem?

@norrisjeremy no, I am aware of the issue and I am following the releases in order to fix them ASAP. The release for version 11 was early this morning and I was waiting for that. If you try again you will be able to get the right archive now. Sorry this is bothering you, I haven't had enough time to work on a fix and take into account that as many other people I am using my free time to do this. The only thing I can do now is to disable releases for mac so doesn't bother users until I publish them manually. Hope you can understand.

marc0der commented 2 years ago

Thanks, @eddumelendez. I think it's preferable to disable these releases for Mac until you have a working fix in place. I'd prefer that above having broken binaries distributed. wdyt?

eddumelendez commented 2 years ago

agree!

eddumelendez commented 2 years ago

sapmachine is proving the right resources in a standard format. see https://github.com/SAP/SapMachine/issues/1024

I think we can close this ticket

adacaccia commented 2 years ago

But it fails for Temurin 11 also, and on Linux Ubuntu 21.04!

sdk install java 11.0.14-tem 
Downloading: java 11.0.14-tem
In progress...
Warning: Failed to open /home/USER/.sdkman/tmp/java-11.0.14-tem.headers.tmp
curl: (23) Failed writing received data to disk/application
grep: /home/USER/.sdkman/tmp/java-11.0.14-tem.headers.tmp: No such file or directory
Download has failed, aborting!
Can not install java 11.0.14-tem at this time...

Looking at that "tmp" folder, shows all of my failed attempts to get some sort of java sdk for Scala working:

ls -l .sdkman/tmp/
total 56
922 mar 23 15:13 hook_post_java_11.0.12-open.sh
919 mar 23 15:25 hook_post_java_11.0.14-tem.sh
916 mar 23 15:09 hook_post_java_17.0.2-tem.sh
156 mar 23 15:13 hook_pre_java_11.0.12-open.sh
155 mar 23 15:25 hook_pre_java_11.0.14-tem.sh
154 mar 23 15:09 hook_pre_java_17.0.2-tem.sh
29865 mar 23 15:08 sdkman-5.14.1.zip

I am using standard regular bash as shell, either. Where to look to dig deeper?

marc0der commented 2 years ago

Hi @adacaccia, I don't think this is the same issue as what you are seeing. This was a specific problem to do with sapmachine.

I'm also a Linux user and don't have any issues installing the 11.0.14-tem release. It looks like your issue is due to some permissions problem:

Warning: Failed to open /home/USER/.sdkman/tmp/java-11.0.14-tem.headers.tmp
curl: (23) Failed writing received data to disk/application
grep: /home/USER/.sdkman/tmp/java-11.0.14-tem.headers.tmp: No such file or directory

Can you try to flush the tmp folder with sdk flush to see if that fixes it? Please let me know.

wellingtonpaim commented 1 year ago

`wellington@wellington-Lenovo-G50-80:~$ sdk install java 19-open

Downloading: java 19-open

In progress...

Warning: Failed to open /home/wellington/.sdkman/tmp/java-19-open.headers.tmp curl: (23) Failed writing received data to disk/application grep: /home/wellington/.sdkman/tmp/java-19-open.headers.tmp: Arquivo ou diretório inexistente Download has failed, aborting!

Can not install java 19-open at this time... `

I have Ubuntu 22.04 and I installed SDKMan, but I can't install any Java jdk from SDKMan, see the error that appears. Can anyone guide me to solve this?

Guki12 commented 1 year ago

@wellingtonpaim did you solve somehow problem? I am getting same error now

gamagabriel commented 3 months ago

Hello, everyone! I have the same problem of @wellingtonpaim, using Ubuntu 22.04 and SDKMan.

gamagabriel commented 3 months ago

Just for information, I resolved this issue after reinstalling cURL, using the following commands:

sudo snap remove curl
sudo apt-get install curl

sdkman / sdkman-db-migrations

Bug: Sap Machine OpenJDK fails to install #496

I have Ubuntu 22.04 and I installed SDKMan, but I can't install any Java jdk from SDKMan, see the error that appears. Can anyone guide me to solve this?