search

watertap-org / watertap

The WaterTAP development repository
https://watertap.readthedocs.io/en/latest
Other
59 stars 58 forks source link

macOS x86_64: Must reinstall ipopt for new development environment in contributing repo #871

Closed kurbansitterley closed 1 year ago

kurbansitterley commented 1 year ago

Description

In creating a new development environment for the SETO project, I was not able to run tests until I reinstalled ipopt.

Expected Behavior

Since I already have a working watertap development environment, I did not think it was necessary to reinstall ipopt on this new environment.

Steps to Reproduce

From the README on the watertap-seto repository:

  1. clone repository
  2. conda create --yes --name watertap-seto-dev-env python=3.10 && conda activate watertap-seto-dev-env
  3. pip install -r requirements-dev.txt
  4. try to run any test that requires model solve
  5. pyomo.common.errors.ApplicationError: No executable found for solver 'ipopt-watertap'

Environment

Anything Else?

This issue is remedied by simply reinstalling ipopt following the directions from the WaterTAP docs for Intel-based Mac:

  1. xcode-select --install (if necessary)
  2. conda install --yes cmake
  3. pyomo build-extensions
  4. conda install --yes -c conda-forge ipopt coincbc
kurbansitterley commented 1 year ago

Notably, I do not have this issue when cloning my existing watertap-dev environment and running tests, e.g.:

conda create --name watertap-dev-test --clone watertap-dev

kurbansitterley commented 1 year ago

Full output after running pyomo build-extensions when reinstalling ipopt if useful:

**** Building AMPL External function demo library ****
-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) 
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/gs/9r53xcnx4kjbgntt31vsh4mjz570mg/T/tmpo7lxri4v
[ 50%] Building C object CMakeFiles/asl_external_demo.dir/functions.c.o
[100%] Linking C shared library libasl_external_demo.dylib
[100%] Built target asl_external_demo
Install the project...
-- Install configuration: ""
-- Installing: /Users/ksitterl/.pyomo/lib/libasl_external_demo.dylib
Installed AMPL External function demo library to /Users/ksitterl/.pyomo

**** Building APPSI ****
ERROR: ModuleNotFoundError: No module named 'pybind11'

**** Building MCPP library ****
ERROR: RuntimeError: Cannot identify the location of the MCPP source
    distribution

**** Building PyNumero libraries ****
-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) 
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) 
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/gs/9r53xcnx4kjbgntt31vsh4mjz570mg/T/tmpl1av_r34
[ 25%] Building CXX object CMakeFiles/pynumero_ASL.dir/AmplInterface.cpp.o
[ 50%] Linking CXX shared library libpynumero_ASL.dylib
[ 50%] Built target pynumero_ASL
[ 75%] Building CXX object tests/CMakeFiles/pynumero_asl_test.dir/simple_test.cpp.o
[100%] Linking CXX executable pynumero_asl_test
[100%] Built target pynumero_asl_test
Install the project...
-- Install configuration: ""
-- Installing: /Users/ksitterl/.pyomo/lib/libpynumero_ASL.dylib
-- Installing: /Users/ksitterl/.pyomo/bin/tests/pynumero_asl_test
-- Installing: /Users/ksitterl/.pyomo/bin/tests/simple_nlp.nl
Installed PyNumero libraries to /Users/ksitterl/.pyomo
INFO: Finished building Pyomo extensions.
INFO: The following extensions were built:
        [ OK ]  ampl_function_demo
        [FAIL]  appsi
        [FAIL]  mcpp
        [ OK ]  pynumero
lbianchi-lbl commented 1 year ago

Thanks for opening this issue and the detailed error report. Without having a specific hypothesis for a cause in mind:

  1. Does it make a difference if idaes get-extensions is run after pip install -r requirements-dev.txt?
  2. What's the output of idaes environment-info?
kurbansitterley commented 1 year ago
  1. I hadn't been running idaes get-extensions because I didn't think it was supported for Mac. Here is the output: 
Getting files...

Unsupported platfrom: darwin-x86_64.

Specify an os with --distro <os>:
  1. output from idaes environment-info: 
IDAES
    Version: 2.0.0.a3
    Git Hash: None
    Binary Directory: /Users/ksitterl/.idaes/bin
    Data Directory: /Users/ksitterl/.idaes
    Global Config: /Users/ksitterl/.idaes/idaes.conf

Pyomo
    Version: 6.4.4

Python
    Version: 3.10.8 (main, Nov 24 2022, 08:09:04) [Clang 14.0.6 ]
    Executable: /Users/ksitterl/opt/anaconda3/envs/watertap-seto/bin/python

OS
    Platform: Darwin
    Release: 21.6.0
    Version: Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64

Dependencies
    backports.shutil-get-terminal-size: 1.0.0
    bunch: 1.0.1
    click: 8.1.3
    colorama: 0.4.6
    distro: 1.8.0
    flask: 2.2.2
    flask-cors: 3.0.10
    ipython: 7.34.0
    jupyter: 1.0.0
    lxml: 4.9.1
    matplotlib: 3.6.2
    nbconvert: 7.2.6
    nbformat: 5.7.0
    networkx: 2.8.8
    numpy: 1.23.5
    omlt: 0.3.1
    openpyxl: 3.0.10
    pandas: 1.5.2
    pint: 0.20.1
    psutil: 5.9.4
    pytest: 7.2.0
    python-slugify: 7.0.0
    pyyaml: 6.0
    rbfopt: 4.2.5
    requests: 2.28.1
    scipy: 1.9.3
    sympy: 1.11.1
    tinydb: 4.7.0
    xlrd: 2.0.1

Extras
    seaborn: Not Installed

Solvers
    bonmin: Not Installed
    cbc: 2.10.5.0
    couenne: Not Installed
    dot_sens: Not Installed
    ipopt: 3.14.10.0
    ipopt_l1: Not Installed
    ipopt_sens: 3.14.10.0
    k_aug: Not Installed
lbianchi-lbl commented 1 year ago
  1. I hadn't been running idaes get-extensions because I didn't think it was supported for Mac. Here is the output:

Ah, I see now that you specified the OS/arch in the issue, but I missed that part, thanks for clarifying. I've change the title to make it clearer that this is about the (experimental and/or unsupported) x86_64 macOS solvers.

zacharybinger commented 1 year ago

Ran into a very similar issue after install resulting in the following error code:

ERROR: Solver (ipopt) returned non-zero return code (-6)
ERROR: Solver log: dyld[11936]: Library not loaded:
    '/opt/homebrew/opt/gcc/lib/gcc/current/libquadmath.0.dylib'
      Referenced from: '/Users/zbinger/.idaes/bin/ipopt' Reason: tried:
      '/libquadmath.0.dylib' (no such file),
      '/Users/zbinger/.idaes/bin/libquadmath.0.dylib' (no such file),
      '/opt/homebrew/opt/gcc/lib/gcc/current/libquadmath.0.dylib' (no such
      file), '/usr/local/lib/libquadmath.0.dylib' (no such file),
      '/usr/lib/libquadmath.0.dylib' (no such file)

Followed this thread and got the following output after running idaes get-extensions

Environment OS: macOS Monterey 12.6.2, MacBookPro 14-inch 2021, Apple M1 Pro Python: 3.8.12 WaterTAP: 0.7.0

Solvers
    bonmin: Not Installed
    cbc: U.n.k.n.o.w.n. .V.e.r.s.i.o.n. .I.n.s.t.a.l.l.e.d
    couenne: Not Installed
    dot_sens: Not Installed
    ipopt: U.n.k.n.o.w.n. .V.e.r.s.i.o.n. .I.n.s.t.a.l.l.e.d
    ipopt_l1: Not Installed
    ipopt_sens: Not Installed
    k_aug: Not Installed

Fixed the issue by installing ipopt directly via anaconda and copying the necessary lib files from the anaconda /lib folder to the .idaes/bin/ folder