Help installing esa_snappy

[jeffdiamonddw]

I have been trying to install esa_snappy for a number of days with no success so far. I need help please. details are below. I have also tried python3.7 and various other things such as using pip to install jpy .

If anyone can help me or provide a script that works I would be very very grateful!

Thankyou!

Jeff Diamond

**I began with this ubuntu on an aws ec2 instance: ubuntu@ip-172-31-15-21:~/snap/bin$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammy

**Then ran the following script:

install python tools

sudo apt update sudo apt-get install -y -o DPkg::Options::=--force-confdef cmake python3-dev python3-pip

install java

cd ~ wget https://download.oracle.com/java/21/latest/jdk-21_linux-x64_bin.deb -O jdk-21_linux-x64_bin.deb sudo apt-get install -y ./jdk-21_linux-x64_bin.deb sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk-21-oracle-x64/bin/java" 2 sudo update-alternatives --set java /usr/lib/jvm/jdk-21-oracle-x64/bin/java export JDK_HOME=$(java -XshowSettings:properties -version 2>&1 | grep "java.home" | tr -s ' ' | cut -d' ' -f4) echo export "JDK_HOME=$JDK_HOME" >> ~/.bashrc echo export "JAVA_HOME=$JDK_HOME" >> ~/.bashrc source ~/.bashrc

install snap

cd wget https://download.esa.int/step/snap/9.0/installers/esa-snap_sentinel_unix_9_0_0.sh -O /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh chmod a+x /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh echo 'export SNAP_HOME=/home/ubuntu/snap' >> ~/.bashrc source ~/.bashrc

install jpy

sudo apt install maven git clone https://github.com/jpy-consortium/jpy.git cd jpy python3 setup.py bdist_wheel find /home/ubuntu/jpy/dist/ -name *.whl | xargs -I{} cp {} /home/ubuntu/.local/lib/python3.7/site-packages/snappy export LD_LIBRARY_PATH=/home/ubuntu/snap/jre/jre/lib/amd64/server/

configure snappy

cd /home/ubuntu/snap/bin ./snappy-conf /usr/bin/python3 /home/ubuntu/.local/lib/python3.10/site-packages/

**** ***I get this error ubuntu@ip-172-31-15-21:~/snap/bin$ ./snappy-conf /usr/bin/python3 /home/ubuntu/.local/lib/python3.10/site-packages/ /home/ubuntu/snap/bin/../platform/lib/nbexec: WARNING: environment variable DISPLAY is not set Configuring SNAP-Python interface... java.io.IOException: Python configuration failed. Command [/usr/bin/python3 ./snappyutil.py --snap_home /home/ubuntu/snap --java_module /home/ubuntu/snap/snap/modules/org-esa-snap-snap-python.jar --force --log_file ./snappyutil.log --jvm_max_mem 10G --java_home /home/ubuntu/snap/jre/jre --req_arch amd64] failed with return code 10. Please check the log file '/home/ubuntu/.local/lib/python3.10/site-packages/snappy/snappyutil.log'. at org.esa.snap.python.PyBridge.configureJpy(PyBridge.java:232) at org.esa.snap.python.PyBridge.installPythonModule(PyBridge.java:149) at org.esa.snap.rcp.cli.SnapArgsProcessor.processPython(SnapArgsProcessor.java:103) at org.esa.snap.rcp.cli.SnapArgsProcessor.process(SnapArgsProcessor.java:49) at org.netbeans.modules.sendopts.DefaultProcessor.process(DefaultProcessor.java:202) at org.netbeans.spi.sendopts.Option$1.process(Option.java:387) at org.netbeans.api.sendopts.CommandLine.process(CommandLine.java:317) at org.netbeans.modules.sendopts.HandlerImpl.execute(HandlerImpl.java:62) at org.netbeans.modules.sendopts.Handler.cli(Handler.java:69) at org.netbeans.CLIHandler.notifyHandlers(CLIHandler.java:234) at org.netbeans.core.startup.CLICoreBridge.cli(CLICoreBridge.java:82) at org.netbeans.CLIHandler.notifyHandlers(CLIHandler.java:234) at org.netbeans.CLIHandler$1.exec(CLIHandler.java:268) at org.netbeans.CLIHandler.finishInitialization(CLIHandler.java:447) at org.netbeans.MainImpl.finishInitialization(MainImpl.java:256) at org.netbeans.Main.finishInitialization(Main.java:92) at org.netbeans.core.startup.Main.start(Main.java:316) at org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:123) at java.lang.Thread.run(Thread.java:748) Python configuration error: Python configuration failed. Command [/usr/bin/python3 ./snappyutil.py --snap_home /home/ubuntu/snap --java_module /home/ubuntu/snap/snap/modules/org-esa-snap-snap-python.jar --force --log_file ./snappyutil.log --jvm_max_mem 10G --java_home /home/ubuntu/snap/jre/jre --req_arch amd64] failed with return code 10. Please check the log file '/home/ubuntu/.local/lib/python3.10/site-packages/snappy/snappyutil.log'.

*** and this is the resulting log file ubuntu@ip-172-31-15-21:~/snap/bin$ cat /home/ubuntu/.local/lib/python3.10/site-packages/snappy/snappyutil.log INFO: Installing from Java module '/home/ubuntu/snap/snap/modules/org-esa-snap-snap-python.jar' WARNING: Architecture requirement possibly not met: Python is x86_64 but JVM requires amd64 INFO: Installing jpy... ERROR: The module 'jpy' is required to run snappy, but no binary 'jpy' wheel matching the pattern 'jpy-{version}-cp310-{abi_tag}-linux_x86_64.whl' could be found. You can try to build a 'jpy' wheel yourself, then copy it into "/home/ubuntu/.local/lib/python3.10/site-packages/snappy", and then run the configuration again. Unzip the jpy sources in /home/ubuntu/.local/lib/python3.10/site-packages/snappy/jpy-.zip, then $ cd jpy- $ python setup.py bdist_wheel $ cp dist/*.whl "/home/ubuntu/.local/lib/python3.10/site-packages/snappy" Or get the source code from https://github.com/bcdev/jpy and follow the build instructions: $ git clone https://github.com/bcdev/jpy.git $ cd jpy ERROR: Configuration failed with exit code 10

[SabineEmbacher]

Dear Jeff,

have you tried to find help for this problem in the forum? E.g. using the expanded search? https://forum.step.esa.int/search?expanded=true&q=snappy

Best regards Sabine

[dolaf]

Hi Jeff,

thanks a lot for your detailed bug report.

From the messages in your 'Install jpy' and 'Configure snappy' steps, it seems that Python versions 3.7 and 3.10 are mixed up, which might be a source of the problem:

python3 setup.py bdist_wheel find /home/ubuntu/jpy/dist/ -name *.whl | xargs -I{} cp {} /home/ubuntu/.local/lib/python3.7/site-packages/snappy export LD_LIBRARY_PATH=/home/ubuntu/snap/jre/jre/lib/amd64/server/

configure snappy

cd /home/ubuntu/snap/bin ./snappy-conf /usr/bin/python3 /home/ubuntu/.local/lib/python3.10/site-packages/

However, in the past we received various feedback from users who had similar problems during the installation, and we agree that this procedure is not very user-friendly. Therefore, the snappy interface has been significantly revised for the new SNAP 10 which will be released very soon. In especially, Python versions up to 3.10 will be supported, and the required jpy wheels will come with the snappy module, thus a manual jpy installation will no longer be needed.

The latest documentation on the current installation procedure (for SNAP <= 9) can be found here.

Also note that the current SNAP-Python interface is called 'snappy', whereas the new, revised module will be called 'esa_snappy' from SNAP 10 onwards (documentation in preparation here).

We hope this will help at least for the future.

Cheers, Olaf

[jeffdiamonddw]

Thanks everyone. I found a solution by Ferni at https://forum.step.esa.int/t/problems-with-snappy-and-python-3-8/36833

This is the resulting script run on ubuntu22:

some apt-get installations

sudo apt update sudo apt-get install -y -o DPkg::Options::=--force-confdef cmake python3-dev python3-pip

install python3.8

sudo apt update sudo apt install -y software-properties-common sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install -y python3.8 sudo apt install -y python3.8-distutils sudo apt install -y python3.8-dev

more apt installations

sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev git

install java

cd ~ wget https://download.oracle.com/java/21/latest/jdk-21_linux-x64_bin.deb -O jdk-21_linux-x64_bin.deb sudo apt-get install ./jdk-21_linux-x64_bin.deb sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk-21-oracle-x64/bin/java" 2 sudo update-alternatives --set java /usr/lib/jvm/jdk-21-oracle-x64/bin/java export JDK_HOME=$(java -XshowSettings:properties -version 2>&1 | grep "java.home" | tr -s ' ' | cut -d' ' -f4) echo export "JDK_HOME=$JDK_HOME" >> ~/.bashrc echo export "JAVA_HOME=$JDK_HOME" >> ~/.bashrc source ~/.bashrc

install esa snap toolbox

cd wget https://download.esa.int/step/snap/9.0/installers/esa-snap_sentinel_unix_9_0_0.sh -O /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh chmod a+x /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh /home/ubuntu/esa-snap_sentinel_unix_9_0_0.sh echo 'export SNAP_HOME=/home/ubuntu/snap' >> ~/.bashrc source ~/.bashrc

download wheel for jpy

wget https://anaconda.org/Terradue/jpy/0.9.0/download/linux-64/jpy-0.9.0-py38_3.tar.bz2 tar -xvf py-0.9.0-py38_3.tar.bz2

run snappy config

cd ~/snap/bin ./snappy-conf /usr/bin/python3.8

copy wheel file to snappy installation and run snappy-conf again

cp ~/jpy_wheel/jpy-0.9.0-cp38-cp38-linux_x86_64.whl ~/.snap/snap-python/snappy/ cd ~/snap/bin ./snappy-conf /usr/bin/python3.8 /home/ubuntu/.local/lib/python3.8/site-packages/

put some environment variables in .bashrc

export PYTHONPATH="${PYTHONPATH}:/home/ubuntu/.snap/snap-python/snappy" echo export PYTHONPATH=${PYTHONPATH}:/home/ubuntu/.snap/snap-python/snappy >> /home/ubuntu/.bashrc export SNAP_HOME=/home/ubuntu/snap >> /home/ubuntu/.bashrc echo export SNAP_HOME=/home/ubuntu/snap >> /home/ubuntu/.bashrc source /home/ubuntu/.bashrc

[dolaf]

Hi Jeff, thanks for the info. Good to hear that the installation sequence now works for you!

[h-a-graham]

Just wanted to add my solution here - I was also having trouble with the python module installation on linux, the following bash script solved it for me:

#!/bin/bash
set -e

# This script builds the ESA SNAP Python API and runs the tests

# Set the paths to:
# ESA SNAP Python API
ESA_SNAPPY="$HOME/.snap/snap-python/esa_snappy"
# ESA SNAP binaries
ESA_SNAP_BIN="$HOME/esa-snap/bin"
# ESA SNAP virtual environment
VENV_LOC="$HOME/virtualenvs/snap"

# Create the virtual environment
if [ ! -d "$VENV_LOC" ]; then
  python -m venv "$VENV_LOC"
fi

# Activate the virtual environment
source "$VENV_LOC/bin/activate"

# Copy the jpy wheel to the binary folder
cp "$ESA_SNAPPY/lib/jpy-0.13.0-cp310-cp310-linux_x86_64.whl" \
  "$ESA_SNAP_BIN/jpy-0.13.0-cp310-cp310-linux_x86_64.whl"

# Copy the python module to the virtual environment
cp -r "$ESA_SNAPPY" "$VENV_LOC"/lib/python*/site-packages

# Run snappy-conf to set up the python module
"$ESA_SNAP_BIN/snappy-conf" "$HOME/virtualenvs/snap/bin/python" "$VENV_LOC"/lib/python*/site-packages

# Install missing dependencies and pytest
pip install numpy
pip install pytest

# Run tests
cd "$VENV_LOC"/lib/python*/site-packages/esa_snappy/tests
pytest

[dolaf]

Hi, thanks for posting this. However, note that with SNAP 10 (released on April 18th) it should no longer be necessary to manually copy jpy wheels or the esa_snappy module. After you selected the Python of your choice (base or a virtual environment), you can directly run snappy-conf, as you did in your script.

The latest how-to is here. We try to frequently improve the documentation, following recent user feedback.

[h-a-graham]

Thanks for the link to those instructions! Just to confirm though that I did attempt that approach first and I believe that I'm running the latest snap version: My OS is Ubuntu 22.04.4 LTS

I'm playing around with making an R wrapper for GPT so I'll have a go at a clean install soon and try linking up python again and report back...

[h-a-graham]

quick update - if I find more time test I will try update further but I suspect the issue may have resulted from using the installer GUI to attempt setting up the python environment in the first instance. When I install snap in headless mode with -q, the snappy-conf util works exactly as expected...