nextflow works with java v17 but not v18

HenriettaHolze commented 2 years ago

Bug report

The get started page states that nextflow can run with Java up to version 18. After installing Java v18 I could not run nextflow but after downgrading to v17 I could run nextflow.

Expected behavior and actual behavior

I would expect nextflow to run but I get the following error after running the nextflow command:

CAPSULE EXCEPTION: Could not parse version: 18.0.1.1 while processing attribute Min-Java-Version: 1.8.0 (for stack trace, run with -Dcapsule.log=verbose)
USAGE: java <options> -jar .nextflow/framework/22.04.0/nextflow-22.04.0-one.jar

Actions:
  capsule.version - Prints the capsule and application versions.
  capsule.modes - Prints all available capsule modes.
  capsule.jvms - Prints a list of all JVM installations found.
  capsule.help - Prints this help message.
  capsule.tree - Prints the capsule's dependency tree.
  capsule.resolve - Downloads all un-cached dependencies.

Options:
  capsule.mode=<value> - Picks the capsule mode to run.
  capsule.reset - Resets the capsule cache before launching. The capsule to be re-extracted (if applicable), and other possibly cached files will be recreated.
  capsule.log=<value> (default: quiet) - Picks a log level. Must be one of none, quiet, verbose, or debug.
  capsule.java.home=<value> - Sets the location of the Java home (JVM installation directory) to use; If 'current' forces the use of the JVM that launched the capsule.
  capsule.java.cmd=<value> - Sets the path to the Java executable to use.
  capsule.jvm.args=<value> - Sets additional JVM arguments to use when running the application.
  capsule.local=<value> - Sets the path of the local Maven repository to use.
Unable to initialize nextflow environment

Steps to reproduce the problem

Installed Java v18 from https://download.oracle.com/java/18/latest/jdk-18_macos-aarch64_bin.dmg .

Installed NextFlow according to get started guide.

Runnig nextflow resulted in above error.

Installed Java v17 and downgraded with export JAVA_HOME=$(/usr/libexec/java_home -v 17).

Now I can run nextflow.

Program output

Above error.

Environment

Nextflow version: 22.04.0.5697
Java version: 18.0.1.1 / 17.0.3.1
Operating system: macOS M1
Bash version: zsh 5.8 (x86_64-apple-darwin21.0)

jorgeaguileraseqera commented 2 years ago

I followed your steps and it's working on my computer with a fresh installation

maybe do you have an old version of nextflow into the PATH ?

pditommaso commented 2 years ago

think it's related to the version number with four digits 18.0.1.1. See https://github.com/nextflow-io/capsule/pull/5

jorgeaguileraseqera commented 2 years ago

I've tried with 18.0.1.1 and it works fine

$ rm -rf /home/jorge/.nextflow

$java -version
openjdk version "18.0.1.1" 2022-04-22
OpenJDK Runtime Environment (build 18.0.1.1+2-6)
OpenJDK 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)

$  ./nextflow -version
CAPSULE: Downloading dependency org.multiverse:multiverse-core:jar:0.7.0
CAPSULE: Downloading dependency ch.qos.logback:logback-classic:jar:1.2.9
CAPSULE: Downloading dependency org.checkerframework:checker-compat-qual:jar:2.0.0
...                                                                     
      N E X T F L O W
      version 22.04.0 build 5697
      created 23-04-2022 18:00 UTC (20:00 CEST)
      cite doi:10.1038/nbt.3820
      http://nextflow.io

pditommaso commented 2 years ago

jorgeaguileraseqera commented 2 years ago

I had the same problem a few weeks ago but was because I have an old version of nextflow in /usr/bin , once removed and updated with the last version all worked fine

HenriettaHolze commented 2 years ago

I installed nextflow today for the first time on a new laptop so it can't be an old version of nextflow.

andreiprodan commented 2 years ago

I'm also having issues installing Nextflow on a fresh MacBook M1 laptop. Tried both Java 18, then removed it and tried Java 17.

(base) ➜  ~ java -version
java version "17.0.3.1" 2022-04-22 LTS
Java(TM) SE Runtime Environment (build 17.0.3.1+2-LTS-6)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.3.1+2-LTS-6, mixed mode, sharing)
(base) ➜  ~

When trying to run Nextflow, I get:

(base) ➜  ~ ./nextflow
Error: cannot find Java or it's a wrong version -- please make sure that Java 7 or higher it's installed
Note: Nextflow is trying to use the Java VM defined by the following environment variables:
 JAVA_CMD: /Library/Java/JavaVirtualMachines/jdk-17.0.3.1.jdk/Contents/Home/bin/java
 JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk-17.0.3.1.jdk/Contents/Home/bin/java

Have not been able to get around this.

I'm runinng the latest MacOS (Monterey 12.3.1) on a M1 MacBook.

ProductName:    macOS
ProductVersion: 12.3.1
BuildVersion:   21E258

pditommaso commented 2 years ago

can you please run this command and copy & paste there the output ?

cat `which nextflow` | grep -E 'JAVA_VER|NXF_VER'

HenriettaHolze commented 2 years ago

I have no idea what happened but now it magically works with java v18. I can't reproduce the error anymore.

andreiprodan commented 2 years ago

I had no problems installing Nextflow via conda (conda installs Java 11 and everything works smoothly). I removed everything (Nextflow, Java, conda) and started from scratch to debug:

1) installed Java 17 (via the installer for MacOS: https://www.oracle.com/java/technologies/downloads/#:~:text=https%3A//download.oracle.com/java/17/latest/jdk%2D17_macos%2Daarch64_bin.dmg

2) installed Nextflow with curl -s https://get.nextflow.io | bash

Nextflow installation throws:

➜  ~ curl -s https://get.nextflow.io | bash
CAPSULE EXCEPTION: Could not parse version: 17.0.3.1 while processing attribute Min-Java-Version: 1.8.0 (for stack trace, run with -Dcapsule.log=verbose)
USAGE: java <options> -jar .nextflow/framework/22.04.0/nextflow-22.04.0-one.jar

Actions:
  capsule.version - Prints the capsule and application versions.
  capsule.modes - Prints all available capsule modes.
  capsule.jvms - Prints a list of all JVM installations found.
  capsule.help - Prints this help message.
  capsule.tree - Prints the capsule's dependency tree.
  capsule.resolve - Downloads all un-cached dependencies.

Options:
  capsule.mode=<value> - Picks the capsule mode to run.
  capsule.reset - Resets the capsule cache before launching. The capsule to be re-extracted (if applicable), and other possibly cached files will be recreated.
  capsule.log=<value> (default: quiet) - Picks a log level. Must be one of none, quiet, verbose, or debug.
  capsule.java.home=<value> - Sets the location of the Java home (JVM installation directory) to use; If 'current' forces the use of the JVM that launched the capsule.
  capsule.java.cmd=<value> - Sets the path to the Java executable to use.
  capsule.jvm.args=<value> - Sets additional JVM arguments to use when running the application.
  capsule.local=<value> - Sets the path of the local Maven repository to use.
Unable to initialize nextflow environment

A 'nextflow' file is created, but when trying to run it (after chmod +x nextflow) the same error as above is thrown.

3) after running export JAVA_HOME=$(/usr/libexec/java_home -v 17), when trying to run ./nextflow again, a bunch of stuff is downloaded and installed:

➜  ~ ./nextflow
CAPSULE: Downloading dependency org.multiverse:multiverse-core:jar:0.7.0
CAPSULE: Downloading dependency ch.qos.logback:logback-classic:jar:1.2.9
CAPSULE: Downloading dependency org.checkerframework:checker-compat-qual:jar:2.0.0
CAPSULE: Downloading dependency org.codehaus.mojo:animal-sniffer-annotations:jar:1.14
CAPSULE: Downloading dependency com.google.errorprone:error_prone_annotations:jar:2.1.3
CAPSULE: Downloading dependency org.codehaus.groovy:groovy-templates:jar:3.0.10
CAPSULE: Downloading dependency com.google.guava:guava:jar:24.1.1-jre
CAPSULE: Downloading dependency org.codehaus.jsr166-mirror:jsr166y:jar:1.7.0
CAPSULE: Downloading dependency io.nextflow:nextflow:jar:22.04.0
CAPSULE: Downloading dependency org.slf4j:jcl-over-slf4j:jar:1.7.32
CAPSULE: Downloading dependency com.beust:jcommander:jar:1.35
CAPSULE: Downloading dependency org.slf4j:jul-to-slf4j:jar:1.7.32
CAPSULE: Downloading dependency com.google.j2objc:j2objc-annotations:jar:1.1
CAPSULE: Downloading dependency ch.grengine:grengine:jar:1.3.0
CAPSULE: Downloading dependency jline:jline:jar:2.9
CAPSULE: Downloading dependency com.github.zafarkhaja:java-semver:jar:0.9.0
CAPSULE: Downloading dependency org.jsoup:jsoup:jar:1.14.3
CAPSULE: Downloading dependency org.codehaus.groovy:groovy-xml:jar:3.0.10
CAPSULE: Downloading dependency javax.mail:mail:jar:1.4.7
CAPSULE: Downloading dependency io.nextflow:nf-httpfs:jar:22.04.0
CAPSULE: Downloading dependency commons-lang:commons-lang:jar:2.6
CAPSULE: Downloading dependency org.iq80.leveldb:leveldb-api:jar:0.12
CAPSULE: Downloading dependency org.codehaus.gpars:gpars:jar:1.2.1
CAPSULE: Downloading dependency org.slf4j:slf4j-api:jar:1.7.32
CAPSULE: Downloading dependency com.google.code.findbugs:jsr305:jar:1.3.9
CAPSULE: Downloading dependency org.objenesis:objenesis:jar:2.1
CAPSULE: Downloading dependency com.esotericsoftware.kryo:kryo:jar:2.24.0
CAPSULE: Downloading dependency org.apache.ivy:ivy:jar:2.3.0
CAPSULE: Downloading dependency org.codehaus.groovy:groovy-json:jar:3.0.10
CAPSULE: Downloading dependency com.googlecode.javaewah:JavaEWAH:jar:1.1.6
CAPSULE: Downloading dependency org.codehaus.groovy:groovy-nio:jar:3.0.10
CAPSULE: Downloading dependency org.codehaus.groovy:groovy:jar:3.0.10
CAPSULE: Downloading dependency org.slf4j:log4j-over-slf4j:jar:1.7.32
CAPSULE: Downloading dependency dev.failsafe:failsafe:jar:3.1.0
CAPSULE: Downloading dependency io.nextflow:nf-commons:jar:22.04.0
CAPSULE: Downloading dependency javax.activation:activation:jar:1.1.1
CAPSULE: Downloading dependency org.yaml:snakeyaml:jar:1.28
CAPSULE: Downloading dependency com.jcraft:jsch:jar:0.1.54
CAPSULE: Downloading dependency org.pf4j:pf4j:jar:3.4.1
CAPSULE: Downloading dependency org.iq80.leveldb:leveldb:jar:0.12
CAPSULE: Downloading dependency ch.qos.logback:logback-core:jar:1.2.9
CAPSULE: Downloading dependency com.jcraft:jzlib:jar:1.1.1
CAPSULE: Downloading dependency com.google.code.gson:gson:jar:2.2.4
CAPSULE: Downloading dependency org.eclipse.jgit:org.eclipse.jgit:jar:5.2.1.201812262042-r
CAPSULE: Downloading dependency org.pf4j:pf4j-update:jar:2.3.0
CAPSULE: Downloading dependency commons-codec:commons-codec:jar:1.10
Usage: nextflow [options] COMMAND [arg...]

If a new terminal session is opened, ./nextflow throws the same "CAPSULE EXCEPTION" error as before

if catwhich nextflow| grep -E 'JAVA_VER|NXF_VER' is run at this stage, the output is:

➜  ~ cat `which nextflow` | grep -E 'JAVA_VER|NXF_VER'
cat: not: No such file or directory
cat: found: No such file or directory
NXF_VER=${NXF_VER:-'22.04.0'}
local version=$(set +u; [[ $NXF_VER ]] && printf "v$NXF_VER" || current_ver)
  rm -rf "$NXF_DIST/$NXF_VER" || exit $?
  [[ -z $NXF_EDGE && $NXF_VER = *-edge ]] && NXF_EDGE=1
  unset NXF_VER
nextflow/nextflow:$NXF_VER nextflow "${args[@]}"
NXF_JAR=${NXF_JAR:-nextflow-$NXF_VER-$NXF_PACK.jar}
NXF_BIN=${NXF_BIN:-$NXF_DIST/$NXF_VER/$NXF_JAR}
NXF_URL=${NXF_URL:-$NXF_BASE/v$NXF_VER/$NXF_JAR}
[[ $NXF_LAUNCHER ]] || NXF_LAUNCHER=${NXF_HOME}/tmp/launcher/nextflow-${NXF_PACK}_${NXF_VER}/${NXF_HOST}
JAVA_VER="$(cat "$JAVA_KEY")"
JAVA_VER="$("$JAVA_CMD" $NXF_OPTS -version 2>&1)"
  echo_red "${JAVA_VER:-Failed to launch the Java virtual machine}"
JAVA_VER=$(echo "$JAVA_VER" | awk '/version/ {gsub(/"/, "", $3); print $3}')
if [[ ! $NXF_VER =~ ([0-9]+)\.([0-9]+)\.([0-9].*) ]]; then
echo_red "Not a valid Nextflow version: $NXF_VER"
if [[ ! $JAVA_VER =~ $version_check ]]; then
if [[ ! $JAVA_VER =~ ^(11|12|13|14|15|16|17|18) ]]; then
  echo_yellow "NOTE: Nextflow is not tested with Java $JAVA_VER -- It's recommended the use of version 11 up to 18\n"
[[ -f $JAVA_VER ]] && echo $JAVA_VER > "$JAVA_KEY"
$JAVA_VER
$NXF_VER
if [[ "$JAVA_VER" =~ ^(9|10|11|12|13|14|15|16|17|18) ]]; then

when export JAVA_HOME=$(/usr/libexec/java_home -v 17) is run again (or sourced from a .zshrc file), nextflow works again.

davidmasp commented 2 years ago

Potentially the worst day to update my java version :facepalm:

Anyway, for what is worth, can confirm @andreiprodan workaround (thanks!)

My steps:

deleted old nextflow executable
Run export JAVA_HOME=$(/usr/libexec/java_home -v 17) before nextflow
re-install newer version
include export JAVA_HOME=$(/usr/libexec/java_home -v 17) in a sourced file (i.e. .zshrc)

pditommaso commented 2 years ago

Unable to replicate:

» java --version
java 18.0.1.1 2022-04-22
Java(TM) SE Runtime Environment (build 18.0.1.1+2-6)
Java HotSpot(TM) 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)
» curl -s get.nextflow.io | bash

      N E X T F L O W
      version 22.04.0 build 5697
      created 23-04-2022 18:00 UTC (20:00 CEST)
      cite doi:10.1038/nbt.3820
      http://nextflow.io

Nextflow installation completed. Please note:
- the executable file `nextflow` has been created in the folder: /Users/pditommaso/Projects/somewhere
- you may complete the installation by moving it to a directory in your $PATH

BTW the best way to install and manage different versions of Java is https://sdkman.io/. Literally onliner.

andreiprodan commented 2 years ago

could it be something linked to the Mac M1 CPU / Monterey OS? In my case it was a fresh out-of-the-box laptop so definitely not somthing caused by an older installation of Nextflow. @davidmasp: it was @HenriettaHolze's fix, I just replicated what she did. @pditommaso: thanks for the sdkman tip, will use that next time!

pditommaso commented 2 years ago

could it be something linked to the Mac M1 CPU

I've tested with the same

pditommaso commented 2 years ago

Unless, it's something related to the default shell. Can you please try

echo $SHELL

andreiprodan commented 2 years ago

echo $SHELL gives/bin/zsh

The zsh version is:

zsh 5.8 (x86_64-apple-darwin21.0)

pditommaso commented 2 years ago

Still not able to replicate it. We need a more precise sequence of steps showing the problem

andreiprodan commented 2 years ago

it's weird indeed.

I decided to try the sdkman java installation (so removed all Java, Nextflow, all modifications to the .zshrc and started from scratch):

1) installed sdkman: curl -s "https://get.sdkman.io" | bash source "$HOME/.sdkman/bin/sdkman-init.sh

2) installed java with sdk: sdk install java

The java version sdk installs is:

(base) ➜  ~ java -version
openjdk version "17.0.2" 2022-01-18
OpenJDK Runtime Environment Temurin-17.0.2+8 (build 17.0.2+8)
OpenJDK 64-Bit Server VM Temurin-17.0.2+8 (build 17.0.2+8, mixed mode)

I notice that the Java 17 version installed by sdk ("17.0.2") is different than the version installed by the Arm 64 DMG Installer (17.0.3.1) https://www.oracle.com/java/technologies/downloads/#:~:text=https%3A//download.oracle.com/java/17/latest/jdk%2D17_macos%2Daarch64_bin.dmg ... which installs this version:

(base) ➜  ~ java -version
java version "17.0.3.1" 2022-04-22 LTS
Java(TM) SE Runtime Environment (build 17.0.3.1+2-LTS-6)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.3.1+2-LTS-6, mixed mode, sharing)

3) install nextflow: curl -s https://get.nextflow.io | bash

When using this installation flow, Nextflow runs without any issues.

pditommaso commented 2 years ago

Does work both with version 17.0.2 and 17.0.3.1 ?

davidmasp commented 2 years ago

I think the 17.0.3.1 version is to blame. I am not sure why it cannot be replicated, I am not familiar enough on how java works.

Listing my steps to replicate the error, just re-tested them now:

Go back to blank state

rm nextflow (rm -f nextflow)
rm jdk install (sudo rm -rf /Library/Java/JavaVirtualMachines/jdk-17.0.3.1.jdk)

This gives me the default (?) java install. java version "1.8.0_311".

Here I can install nextflow correctly (version 22.04.0). I do get this note message though: NOTE: Nextflow is not tested with Java 1.8.0_311 -- It's recommended the use of version 11 up to 18.

When trying to launch hello pipeline I get another error about the /Library/Internet: No such file or directory which I was getting sporadically before too, now I seem to get it all the time.

Anyway, not clear why this doesn't work now. I think I might have had nf v.21 before (?).

Install java 17 from the oracle website (see here). I get the DMG installer. Execute and go through the installation dialogs.

I got the 17 instead of 18 because it is supposed to be the LTS.

This gives me the java version "17.0.3.1" 2022-04-22 LTS

(reset the terminal)

Install nextflow with the one-liner from the website and end up with:

CAPSULE EXCEPTION: Could not parse version: 17.0.3.1 while processing attribute Min-Java-Version

rm nextflow, export the export JAVA_HOME=$(/usr/libexec/java_home -v 17) and try again

nextflow installs well and I am able to run the nextflow run hello instantly. Note that whenever I reset the terminal it fails again with the same issue as before, so I need to set the JAVA_HOME variable in a bashrc-like file.

(other info, can update if needed)

echo $SHELL
/bin/zsh

I hope this helps in any way. I will definitely try the SDKMAN thing, seems quite intuitive, thanks for the tip!

EDIT: I did also test removing the executable nextflow and the main directory ~/.nextflow. Both gave me the same error.

pditommaso commented 2 years ago

If so, it should be solved by this.

It will be included with the next release https://github.com/nextflow-io/nextflow/commit/20ec16977a7eda613d59230b0d55379329c4311a

pditommaso commented 2 years ago

This should be solved by latest version 22.05.0-edge

odagayev commented 2 years ago

Not sure if this was related or not but I ran into similar issues installing Nextflow on a brand new Macbook Pro M1 due to issues with JAVA.

The practical solution was to use SDKman (https://sdkman.io/) - as suggested by @ewels in the Nextflow slack channel.

curl -s "https://get.sdkman.io/" | bash sdk init sdkman install java

And then to re-download the Nextflow installation and add it to your path. :)

Hope this helps.

rwinand commented 2 years ago

Just leaving a comment here in case someone else runs into this issue when using a different installation method for Java.

I ran into the same issue on Linux but I am not using a .deb installation, I am running Java extracted from the Linux x64 tar.gz file on the Oracle website.

Based on the previous answers, I found that the issue could be solved by adding the location of the extracted archive to the JAVA_HOME variable:

export JAVA_HOME=/usr/local/bin/jdk-18.0.1.1

telatin commented 2 years ago

I'm still fiddling with this and will try some of the suggestions, but just to report that on a new and "clean"ish Mac M1 with a fresh Java (apparently they now ship a universal binary) installation I got the same problem

$ java --version
java 18.0.1.1 2022-04-22
Java(TM) SE Runtime Environment (build 18.0.1.1+2-6)
Java HotSpot(TM) 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)

$ which java
/usr/bin/java

$ file $(which java)
/usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64
- Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e
- Mach-O 64-bit executable arm64e]
/usr/bin/java (for architecture x86_64):    Mach-O 64-bit executable x86_64
/usr/bin/java (for architecture arm64e):    Mach-O 64-bit executable arm64e

This happens both on a native terminal and on a rosetta emulated one.

yaniv-scala commented 2 years ago

I had the same problem (with new MAC M1 and newly installed Java 18), tried several of the suggested solutions, what worked for me is simply running export JAVA_HOME=$(/usr/libexec/java_home -v 18)

mhebrard commented 2 years ago

Can we have this issue re-open ... after a fresh installation of java on my MacOS M1 using brew, I have

java -version
openjdk version "18.0.2.1" 2022-08-18
OpenJDK Runtime Environment Homebrew (build 18.0.2.1+0)
OpenJDK 64-Bit Server VM Homebrew (build 18.0.2.1+0, mixed mode, sharing)

and still get

wget -qO- https://get.nextflow.io | bash
CAPSULE EXCEPTION: Could not parse version: 18.0.2.1 while processing attribute Min-Java-Version: 1.8.0 (for stack trace, run with -Dcapsule.log=verbose)

seem like nextflow installer still crash when java version got 4 digits

glichtenstein commented 2 years ago

Just posting a solution that worked for me when encountering the same problem. I used conda to create a Nextflow environment and then installed OpenJDK 17 on it, the downloading of nextflow completed and the bash installation worked fine this time.

conda create -n nextflow openjdk=17.0.3 -y; 
conda activate nextflow;
curl -s https://get.nextflow.io | bash

Nextflow installation completed

N E X T F L O W version 22.04.5 build 5708 created 15-07-2022 16:09 UTC (12:09 EDT) cite doi:10.1038/nbt.3820 http://nextflow.io

lucacozzuto commented 2 years ago

Hi, I run in the same problem. I found putting

export JAVA_HOME=$(/usr/libexec/java_home -v 17)

in my .bash_profile. Sourcing it, removing the nextflow executable, and downloading again.

rayhendricks commented 2 years ago

Quick fix for M1 macbook that I have been using is to uninstall homebrew java/jdk use (sdkman)[https://sdkman.io/] and run the following:

brew install docker -cask -- to install docker-desktop needed by nextflow containierization

curl -s "https://get.sdkman.io" | bash -- install sdkman

then

source "/Users/rayhendricks/.sdkman/bin/sdkman-init.sh -- source shell

then

https://get.nextflow.io | bash -- install nextflow

Krithika-Bhuvan commented 1 year ago

Having the same error. Intel mac. Will try with conda

scottgigante-immunai commented 1 year ago

@pditommaso this is still a problem with nextflow 23.04.0

bounlu commented 5 months ago

Now it does not work with Java 22:

$ ./run_sarek.sh
ERROR: Cannot find Java or it's a wrong version -- please make sure that Java 8 or later (up to 21) is installed
NOTE: Nextflow is trying to use the Java VM defined by the following environment variables:
 JAVA_CMD: /opt/anaconda3/envs/java/bin/java
 JAVA_HOME:

pditommaso commented 5 months ago

https://github.com/nextflow-io/nextflow/pull/4970

nextflow-io / nextflow