csu-hmc / opty

A library for using direct collocation in the optimization of dynamic systems.
http://opty.readthedocs.io
Other
86 stars 20 forks source link

Use env.yml in CI and build docs, fixes #155. #158

Closed moorepants closed 1 month ago

moorepants commented 1 month ago

Seems like macosx latest is an arm architecture and there is no openmp available. Need to revert to intel architecture.

moorepants commented 1 month ago

Not it seems that the last successful CI runs were on:

Current runner version: '2.316.1'
Operating System
  macOS
  14.4.1
  23E224
Runner Image
  Image: macos-14-arm64
  Version: 20240422.3
  Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20240422.3/images/macos/macos-14-arm64-Readme.md
moorepants commented 1 month ago
Can we conda create (from explicit)?
  Can we conda create (simple)?
  Can we conda env update?
  ... will conda env update.
  Making patched copy of 'environment-file: opty-dev-env.yml'
  Using: /Users/runner/work/opty/opty/setup-miniconda-patched-opty-dev-env.yml
  name: opty-dev
  channels:
    - conda-forge
  dependencies:
    - coverage
    - cyipopt >=1.1.0
    - cython >=0.29.19
    - matplotlib >=3.2.0
    - numpy >=1.19.0
    - numpydoc
    - openmp
    - pytest
    - pytest-cov
    - scipy >=1.5.0
    - sphinx
    - sphinx-gallery
    - sympy >=1.6.0
    - python=3.11

Updating 'opty-dev' env from conda env update...
  /Users/runner/miniconda3/condabin/mamba env update --name opty-dev --file /Users/runner/work/opty/opty/setup-miniconda-patched-opty-dev-env.yml
  Channels:
   - conda-forge
   - defaults
  Platform: osx-arm64
  Collecting package metadata (repodata.json): ...working... done
  Solving environment: ...working... failed
  Warning: 
  PackagesNotFoundError: The following packages are not available from current channels:

    - openmp

  Current channels:

    - https://conda.anaconda.org/conda-forge/osx-arm64
    - https://repo.anaconda.com/pkgs/main/osx-arm64
    - https://repo.anaconda.com/pkgs/r/osx-arm64

  To search for alternate channels that may provide the conda package you're
  looking for, navigate to

      https://anaconda.org/

  and use the search bar at the top of the page.
moorepants commented 1 month ago

The last working CI with mac does display this warning when running the tests:

opty/tests/test_utils.py::test_ufuncify_matrix
  /Users/runner/work/opty/opty/opty/utils.py:572: UserWarning: openmp is not installed or not working properly, request for parallel execution ignored.
    warnings.warn(msg)
moorepants commented 1 month ago

But mamba list did show:

openmp                    8.0.1                         0    conda-forge
moorepants commented 1 month ago

Looks like the openmp package was last uploaded 4 years ago:

https://anaconda.org/conda-forge/openmp/files

and the openmp feedstock points to an llvm-openmp

https://github.com/conda-forge/openmp-feedstock

Maybe the dependency is supposed to be llvm-openmp.

moorepants commented 1 month ago

If I include "openmp" if fails on mac and if I include "llvm-openmp" it fails on windows. It used to work on all OS with just "openmp". I switched to using an env.yml installed of the mamba install ... command. Maybe the old working mac builds were just installing the old 5 year old openmp package and now it won't try to install that.

moorepants commented 1 month ago

Maybe the old working mac builds were just installing the old 5 year old openmp package and now it won't try to install that.

No, this is from the old mac build:

llvm-openmp               8.0.1                h770b8ee_0    conda-forge
openmp                    8.0.1                         0    conda-forge

But the warning that seemed like openmp wasn't properly installed (compile didn't work in opty) was present there.

moorepants commented 1 month ago

So I just tried to run the exact old CI setup here https://github.com/csu-hmc/opty/pull/160 and it works for all OSes. So it seems that mamba install ... works but using the commands from setup-miniconda from the env file does not.

moorepants commented 1 month ago

Interesting, so the issue was that I switched to setup-miniconda@v3 instead of v2 in the old CI. Maybe it has something to do with: https://github.com/conda-incubator/setup-miniconda/issues/312

moorepants commented 1 month ago

May be similar issue as: https://github.com/tudat-team/tudat/issues/212

moorepants commented 1 month ago

Some other notes:

moorepants commented 1 month ago

It was reported at the openmp feedstock that the package name changed to llvm-openmp and I should be using that.

moorepants commented 1 month ago

Notes on openmp: https://conda-forge.org/docs/maintainer/knowledge_base/#openmp