Install Arosics on Google Colab

[italomira13]

Hello Daniel,

How are you? Today, I'm attempting to install Arosics, but I encountered the following error message:

 _Using cached GDAL-3.8.5.tar.gz (802 kB)
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details._

This issue is hindering our progress on the project for our master's degree at INPE - Brazil. Could you please assist us? Thank you very much!

[danschef]

Hi @italomira13,

thanks for reporting. Could you please add some more details about how exactly you are attempting to install AROSICS and on which platform? Did you follow the installation instructions and are you trying to update a previous installation or to install it from scratch? Please post the entire command line output here so that I can see where it hangs.

[besmets]

Dear Mr. Scheffler,

I also use AROSICS in a Jupyter notebook, on Google Colab, to allow my students to test AROSICS without the need to install a specific Python environment on their computer.

Recently, some students faced an issue while installing AROSICS with pip (conda does not work in Google Colab). The issue actually comes from GDAL, with the following error:

Collecting gdal (from arosics)
  Using cached GDAL-3.9.0.tar.gz (839 kB)
  Installing build dependencies ... done
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

When GDAL 3.9.0 is installed from the source (the above error seems to come from a missed sub-directory in the path, i.e., "https://download.osgeo.org/gdal**/3.9.0/**gdal-3.9.0.tar.gz" instead of "https://download.osgeo.org/gdal/gdal-3.9.0.tar.gz"), AROSICS still does not work and provides this issue that appears when we do the import "from arosics import COREG_LOCAL":

ImportError               Traceback (most recent call last)
/us/local lib/python3.10/dist-packages/osgeo/__init__.py in swig_ import_ helper()
          29     try:
- - ->  30          return importlib. import_ module(mname)
          31     except ImportError:

---------- 13 frames ----------

ImportError: /usr/local/lib/python3.10/dist-packages/osgeo/_gdal.cpython-310-x86_64-linux-gnu.so: undefined symbol: GDALGetSubdatasetInfo

During handling of the above exception, another exception occurred:

ModuleNotFoundError               Traceback (most recent call last)
/usr/lib/python3.10/importlib/__init__.py in import_module(name, package)
          124                         break
          125                  level += 1
          126     return _bootstrap._gcd_import(name[level:], package, level)
          127
          128

ModuleNotFoundError: No module named '_gdal'

Do you have any clue on how to solve this? Is it related to the way AROSICS and GDAL interact?

I thank you very much in advance

Best regards,

Benoît Smets

[danschef]

Hi Benoît,

AROSICS just has GDAL in the requirements, so running the setup procedure will at some point also attempt to install GDAL if it is not already installed in the current environment. If the installation of GDAL fails (as in your case), AROSICS will not be importable because when importing AROSICS, also GDAL is imported.

The above error looks like it is directly coming from the GDAL installation and has nothing to do with AROSICS. Therefore, the issue should be reported in the GDAL issue tracker. You may try to install an older version of GDAL, e.g., by running pip install "gdal<3.9"

Best, Daniel

[besmets]

Many thanks for this fast reply! I will try with another version of gdal.

All the best,

Benoît

[italomira13]

Hello Benôit Smets, are you good?

I had the same problem. The solution I found was to use AROSICS in ANACONDA, specifically with Jupyter Notebook. I believe the error is related to the order in which the libraries are installed.

Best Regards

[besmets]

Yes that's the solution I was about to propose to the students.

Thanks for the advice!

All the best,

Benoît

[danschef]

There was recently a pull request from the community which added the hint

The gdal library must be installed before numpy, otherwise do a re-installation in order to respect the build isolation.

to the pip installation instructions. See https://github.com/GFZ/arosics/pull/37 for details. Probably that also helps. Thanks @italomira13 for pointing to the installation order.

[danschef]

@italomira13 @besmets
If the issue is solved with that, please feel free to close the issue.

[Vierod]

Hi I was also running into this issue, the confusion is that Google Colab already has GDAL 3.6.4 and it's not being detected or a package is trying to update it as the default behaviour.

You can reproduce the error by trying to force an install of gdal 3.9.0 in Colab using !pip install gdal==3.9.0.

Similarly you can solve it by forcing gdal as the version installed on Colab, so !pip install arosics gdal==3.6.4.

Hopefully this helps!

Best, Alex

[danschef]

Please note that arosics depends on py_tools_ds which (in its latest version) depends on gdal>=3.8.

[Vierod]

Thanks, it does seem to be working without any issues for me on Colab currently using the COREG and DESHIFTER classes and the install method mentioned above.

Checking the pip output it's currently working with py_tools_ds-0.21.1.

It looks like it attempts to install 22.1 and then drops to 21.1 -

Collecting py_tools_ds>=0.18.0 (from arosics)
  Downloading py_tools_ds-0.22.1.tar.gz (89 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.7/89.7 kB 11.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
  Downloading py_tools_ds-0.22.0.tar.gz (89 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 5.5 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
  Downloading py_tools_ds-0.21.1.tar.gz (87 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 6.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done

Edit - so I'd guess the correct pip call for Colab is !pip install arosics gdal==3.6.4 py_tools_ds==0.21.1

[danschef]

Edit - so I'd guess the correct pip call for Colab is !pip install arosics gdal==3.6.4 py_tools_ds==0.21.1

Does this mean that this fixes the issue for you?

[besmets]

I just tested that pip call in my Google Colab notebook and it fixed the issue on my side. Many thanks!