pip install on Colab fails

[jecampagne]

Hi, I've tried to install on Colab

! pip install git+https://github.com/GalSim-developers/GalSim.git

this leads to the following error. (Notice that I have tried on different branch/tags w/o any success)

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting git+https://github.com/GalSim-developers/GalSim.git
  Cloning https://github.com/GalSim-developers/GalSim.git to /tmp/pip-req-build-8fq6_zg1
  Running command git clone --filter=blob:none --quiet https://github.com/GalSim-developers/GalSim.git /tmp/pip-req-build-8fq6_zg1
  Resolved https://github.com/GalSim-developers/GalSim.git to commit 35bb6a03a0f7c03454e783f109f4e2b8d2bf6838
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[rmjarvis]

I don't know anything about Collab. But this is what I get for that command on my laptop:

$ pip install git+https://github.com/GalSim-developers/GalSim.git
Collecting git+https://github.com/GalSim-developers/GalSim.git
  Cloning https://github.com/GalSim-developers/GalSim.git to /private/var/folders/h8/0h3_h20d4jdg8pk8fzwxmr380000gn/T/pip-req-build-wn4eeyuc
  Running command git clone --quiet https://github.com/GalSim-developers/GalSim.git /private/var/folders/h8/0h3_h20d4jdg8pk8fzwxmr380000gn/T/pip-req-build-wn4eeyuc
  Resolved https://github.com/GalSim-developers/GalSim.git to commit 35bb6a03a0f7c03454e783f109f4e2b8d2bf6838
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pybind11>=2.2 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from GalSim==2.4.8) (2.10.3)
Requirement already satisfied: numpy>=1.17 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from GalSim==2.4.8) (1.24.2)
Requirement already satisfied: LSSTDESC.Coord in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from GalSim==2.4.8) (1.2.3)
Requirement already satisfied: astropy in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from GalSim==2.4.8) (5.2.1)
Requirement already satisfied: setuptools>=38 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from GalSim==2.4.8) (67.6.0)
Requirement already satisfied: pyerfa>=2.0 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from astropy->GalSim==2.4.8) (2.0.0.1)
Requirement already satisfied: PyYAML>=3.13 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from astropy->GalSim==2.4.8) (6.0)
Requirement already satisfied: packaging>=19.0 in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from astropy->GalSim==2.4.8) (23.0)
Requirement already satisfied: cffi in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from LSSTDESC.Coord->GalSim==2.4.8) (1.15.1)
Requirement already satisfied: future in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from LSSTDESC.Coord->GalSim==2.4.8) (0.18.3)
Requirement already satisfied: pycparser in /Users/Mike/mambaforge/envs/py3.10/lib/python3.10/site-packages (from cffi->LSSTDESC.Coord->GalSim==2.4.8) (2.21)
Building wheels for collected packages: GalSim
  Building wheel for GalSim (pyproject.toml) ... done
  Created wheel for GalSim: filename=GalSim-2.4.8-cp310-cp310-macosx_11_0_arm64.whl size=4246350 sha256=b2012bf7a2d8d4077474a19c3ec2b70d3f2996470bbb798e3cc0700bdf79db48
  Stored in directory: /private/var/folders/h8/0h3_h20d4jdg8pk8fzwxmr380000gn/T/pip-ephem-wheel-cache-xct0cnq8/wheels/0f/c6/72/9ba9faaaae92dd51dcb22f26d89b5161a29d2702a816b7665a
Successfully built GalSim
Installing collected packages: GalSim
  Attempting uninstall: GalSim
    Found existing installation: GalSim 2.4.7
    Uninstalling GalSim-2.4.7:
      Successfully uninstalled GalSim-2.4.7
Successfully installed GalSim-2.4.8

Do you have all the prereq's installed? pip isn't always great at doing that. And fftw can't be installed via pip, so that needs to already be there.

[jmeyers314]

colab is sorta like jupyter hub from google. It's nice sometimes for sharing notebooks and leaving all the dependency stuff to google (or handling deps with !pip install ... directly inside the notebook).

I think to make this work in the present case, you just need to add !apt-get install libfftw3-dev to make sure FFTW is available before compiling GalSim.

Note that if your use case can handle it, it'll be faster to use one of the precompiled wheels on pypi to install a released version of GalSim. This would also eliminate the need for installing FFTW first.

[rmjarvis]

Jean-Eric, did Josh's suggestion work for you? Is there more to be done here, or can this be closed?

[EiffL]

I think in all cases you should install galsim with:

pip install galsim

The reason is that this will actually grab the pre-compiled wheel, so no need to do any compilations.

Installing galsim from github should only be done if you want to modify galsim and/or want to have to deal with manually compiling it.

I just tried it on colab, works perfectly fine:

notebook here: https://colab.research.google.com/drive/1iUFZ9IzjU0LJX8-dcGgFfeuKoPHvauRc?usp=sharing

[rmjarvis]

OK. It sounds like this issue can be closed. Please reopen if you think there is something to be done on our end.

[jecampagne]

Hi the Galsim install using the suggestion of @EiffL who invite me to use, is fine of course considering the actual pypip version provided. But, I was concerning to use the "main" branch of Galsim to test the maxK computation in Moffat profile using a method that is not avaliable in the current 2.4.11 version. Now, looking at some stackoverflow discussion it may happen that the pb originates in the requirement file....

[jmeyers314]

Hey @jecampagne ,

I think you just need to deal with the FFTW dependency (as described above). If I do

!apt-get install libfftw3-dev
!pip install git+https://github.com/GalSim-developers/GalSim.git@v2.4.11

on colab, things seem to be working.