omas install fails from a conda .yml file.

[cianciosa]

Take the simple .yml file to define a Conda environment.

name: omas
dependencies:
  - pip
  - pip:
    - omas

When building this environment, the pip command to install omas fails when trying to install the pygacode dependency.

conda env create -f omas.yml --force
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: / Ran pip subprocess with arguments:
['/global/homes/m/markcian/miniconda3/envs/omas/bin/python', '-m', 'pip', 'install', '-U', '-r', '/global/u2/m/markcian/ips-fastran/install/conda/condaenv.irdo0hc6.requirements.txt']
Pip subprocess output:
Collecting omas
  Using cached omas-0.87.0.tar.gz (30.4 MB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting numpy>=1.16.1
  Using cached numpy-1.23.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting uncertainties
  Using cached uncertainties-3.1.7-py2.py3-none-any.whl (98 kB)
Collecting pint
  Using cached Pint-0.19.2.tar.gz (292 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting netCDF4
  Using cached netCDF4-1.6.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)
Collecting boto3
  Using cached boto3-1.24.87-py3-none-any.whl (132 kB)
Collecting matplotlib
  Using cached matplotlib-3.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.8 MB)
Collecting scipy
  Using cached scipy-1.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.9 MB)
Collecting h5py
  Using cached h5py-3.7.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
Collecting pymongo
  Using cached pymongo-4.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
Collecting dnspython
  Using cached dnspython-2.2.1-py3-none-any.whl (269 kB)
Collecting xmltodict
  Using cached xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)
Collecting xarray
  Using cached xarray-2022.9.0-py3-none-any.whl (943 kB)
Requirement already satisfied: setuptools>=41.2 in /global/u2/m/markcian/miniconda3/envs/omas/lib/python3.10/site-packages (from omas->-r /global/u2/m/markcian/ips-fastran/install/conda/condaenv.irdo0hc6.requirements.txt (line 1)) (63.4.1)
Collecting tqdm
  Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting Cython
  Using cached Cython-0.29.32-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
Collecting omfit_classes
  Using cached omfit_classes-3.2022.40.2.tar.gz (2.7 MB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pexpect
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting fortranformat
  Using cached fortranformat-1.2.2.tar.gz (22 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pygacode
  Using cached pygacode-0.71.tar.gz (86 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

Pip subprocess error:
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-4impyobm/pygacode_0aafe73b6dbd4533b0f94c796baa2bca/setup.py", line 40, in <module>
          from numpy.distutils.core import setup, Extension
      ModuleNotFoundError: No module named 'numpy'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

failed

CondaEnvException: Pip failed

[orso82]

thoughts @smithsp @kalling ?

[smithsp]

I would guess that the pygacode package is not specifying a numpy dependence, but odd that it was "collected" up above (but maybe not installed yet).

[cianciosa]

I have tried explicitly setting numpy before omas but it fails in the same manner.

On Oct 6, 2022, at 4:35 AM, Sterling Smith @.**@.>> wrote:

I would guess that the pygacode package is not specifying a numpy dependence, but odd that it was "collected" up above (but maybe not installed yet).

— Reply to this email directly, view it on GitHubhxxps://github.com/gafusion/omas/issues/213#issuecomment-1269586917, or unsubscribehxxps://github.com/notifications/unsubscribe-auth/ADAPYD24EGFFCIVP4ODPLBTWB2FLRANCNFSM6AAAAAAQ53PUSM. You are receiving this because you authored the thread.Message ID: @.***>

[orso82]

@cianciosa can you perhaps try to just install pygacode and see if that has the same issue?

The pygacode dependency is not a hard requirement. It is really there only to allow some of the OMAS examples to run.

[cianciosa]

@orso82 Trying just

name: omas
dependencies:
  - pip
  - pip:
    - pygacode

Shows the same problem so it looks like it's not an omas problem but a problem in the pygacode dependency.

[orso82]

Ok. Thanks. @smithsp @kalling could you please look into it ? Independently of that, I should also re-evaluate if omas should really have pygacode as a hard dependency...

[kripnerl]

Hi, the same problem here, when installing OMAS. The problem is that pygacode is using (l. 40)

from numpy.distutils.core import setup, Extension

in the setup.py file. This is being run before other installations. The solution may be found here: https://numpy.org/doc/stable/reference/distutils.html

Thank you @hetsko, for pointing this out.

[hetsko]

Hey, it seems that numpy in pygacode setup really is required (due to fortran code) so migrating to a standard setuptools.setup would not be enough here. And the issue of "build-time dependency in python" is somewhat described here:

https://stackoverflow.com/questions/60045913/installing-numpy-before-using-numpy-distutils-core-setup

A quick and perhaps obvious note for anyone else running into this: a simple workaround for now seems to be to install numpy and omas sequentially in two different pip or conda calls.

pip install numpy
pip install omas

[orso82]

@hetsko thank you for pointing out the origin of the issue and the associated workaround! I now see the problem, that the setup.py file of pygacode requires numpy to run. Perhaps @smithsp or @kalling have some idea about how to bootstrap the pygacode install when starting from a clean slate?

[kalling]

I think another workaround is using the conda package instead instead of using the pip section in a conda yaml , since the conda package should have it pre-built.

[smithsp]

I pushed something in https://github.com/gafusion/gacode/pull/279

[github-actions[bot]]

Stale issue message

[ZedThree]

I'm afraid this is still a problem for pip, I'm not sure about conda. The fix is to add numpy as a build requirement (not just an install requirement!) in pyproject.toml (in pygacode -- that repo is private, which is why I'm making a comment here):

[build-system]
requires = [
    "setuptools < 64",
    "wheel >= 0.29.0",
    "numpy >= 1.20.3",
]
build-backend = "setuptools.build_meta"

This will then let the build tool install numpy before trying to build pygacode.

Is pygacode used in this package? Outside of setup.py I couldn't see any uses of it. So the other fix would be to just delete it from the install requirements here :slightly_smiling_face: