[Python 3.9] Overhaul RMG-RMS Python-Julia dependencies (`juliaup` and `pythoncall`), Add Tests for Arkane Network Algorithms

[JacksonBurns]

Replace #2635, targeting a smaller improvement in Python minor version to keep better compatibility with our dependencies.

[JacksonBurns]

I'll update this comment with the path forward for each platform as the errors roll in.

All Platforms

There are Cython errors all over the place, namely because we have incremented Cython to: cython 3.0.10 from cython 0.29.33 ...oh boy

M1 Macs

The following packages are outright missing, but without any conflicts:

    - nothing provides requested rmg::symmetry
    - nothing provides requested rmg::pyrms
    - nothing provides requested rmg::pydqed >=1.0.3
    - nothing provides requested rmg::pydas >=1.0.3
    - nothing provides requested conda-forge::muq
    - nothing provides requested conda-forge::julia =1.9.1*
    - nothing provides requested rmg::diffeqpy
    - nothing provides julia >=1.5.0 needed by pyjulia-0.6.0-pyhd8ed1ab_0

Anything in the RMG channel we can build for M1 macs, if it comes to that. muq and julia and pyjulia are out of our hands, though.

Ubuntu

We will need to rebuild the rmg channel packages (or just rebase this branch onto #2640, to at least remove a couple of them)

Intel Macs

Same as Ubuntu.

[rwest]

Julia and Pyjulia would be taken care of by #2640 (which would be unblocked if we could get past Python 3.7), so that leave muq? (and the ones that are our responsibility)

[rwest]

Is this faring any better than 3.10 or 3.11? at least it solved the boost - rdkit - muq issue? or do we need to await results on non-M1-platforms?

[JacksonBurns]

Local testing shows that this resolves the muq + rdkit issue. I tried leaving in our packages (rmg channel) to see if the conflict resolution would work, to no avail. I have removed them now to let it (hopefully) suggest solutions to conflicts (if there are any) and then we should have an idea of the path forward.

[JacksonBurns]

@rwest initial tests are done. M1 macs are missing a number of packages that are outside of our control. I think that is a dead end.

Ubuntu and MacOS need the rmg channel packages. If we rebase this on #2640 we can avoid building those we plan to immediately remove afterward (I want to do this).

All platforms will need Cython errors fixed.

[JacksonBurns]

Following some offline discussion, @hwpang and I have agreed that I will rebase this PR onto #2640 and then merge it into the same. This will:

• prevent me having to build packages which we will immediately remove, simplifying this PR: https://github.com/ReactionMechanismGenerator/conda-recipes/pull/18
• require resolving the Cython compilation errors. This may or may not be very difficult - subsequent work in this PR will focus on addressing the scope of these changes.
• still require building pydas and pydqed, which may or not be difficult, but will handled in separate PRs on their respective repositories.

[JacksonBurns]

@hwpang your PR is actually on your fork of RMG, so the PR to implement Python 3.9 would show up on your RMG. I have instead rebased this PR onto your branch, then rebased this branch onto main, and left this PR open again RMG/main. We can either continue all the development here and close yours, move yours to a branch on RMG, or just open the PR on your RMG.

[hwpang]

@hwpang your PR is actually on your fork of RMG, so the PR to implement Python 3.9 would show up on your RMG. I have instead rebased this PR onto your branch, then rebased this branch onto main, and left this PR open again RMG/main. We can either continue all the development here and close yours, move yours to a branch on RMG, or just open the PR on your RMG.

Ah ok, yeah let's just work on this PR since it's more straight forward. We can close my PR for now.

[JacksonBurns]

PyDQED and PyDAS are now built on M1 Macs, Intel Macs, and Linux for Python 3.9 to 3.12 and Numpy 1.21 to 2.0 (as far back each Python version supports) totaling 60 binaries for each package. I implemented automated builds and fixed up the repositories in https://github.com/ReactionMechanismGenerator/PyDQED/pull/6 and https://github.com/ReactionMechanismGenerator/PyDQED/pull/7 and then https://github.com/ReactionMechanismGenerator/PyDAS/pull/15.

We should now be able to continue work on this PR.

[JacksonBurns]

Looks good, just Cython errors now. Best to resolve locally.

Also need to update the build docs action to specify the correct version of Python.

[JacksonBurns]

@hwpang I have resolved the dependency and Cython errors, the CI is now erroring on the Julia stuff - can you take a look?

[hwpang]

@JacksonBurns Thanks for tagging me. It seems that the Julia part fails at resolving some conda packages (see error traces below), are you familiar with any of these incompatible package error?

Looking for: ["libstdcxx-ng[version='>=3.4,<13.0']", "matplotlib[version='>=1']", 'python==3.9', "conda-forge::python[version='>=3.8,<4',build=*cpython*]"]

Could not solve for environment specs
The following packages are incompatible
├─ libglib is installable with the potential options
│  ├─ libglib [2.68.4|2.70.0|...|2.80.2] would require
│  │  └─ libffi [>=3.4,<4.0a0 |>=3.4.2,<3.5.0a0 ], which can be installed;
│  ├─ libglib 2.64.6 would require
│  │  └─ glib 2.64.6 *_0, which can be installed;
│  ├─ libglib 2.66.1 would require
│  │  └─ glib 2.66.1 *_1, which can be installed;
│  ├─ libglib 2.66.2 would require
│  │  └─ glib 2.66.2 *_0, which can be installed;
│  ├─ libglib 2.66.3 would require
│  │  └─ glib 2.66.3 *_1, which can be installed;
│  ├─ libglib 2.66.3 would require
│  │  └─ glib 2.66.3 *_0, which can be installed;
│  ├─ libglib 2.66.4 would require
│  │  └─ glib 2.66.4 *_0, which can be installed;
│  ├─ libglib 2.66.4 would require
│  │  └─ glib 2.66.4 *_1, which can be installed;
│  ├─ libglib 2.66.4 would require
│  │  └─ glib 2.66.4 *_2, which can be installed;
│  ├─ libglib 2.66.5 would require
│  │  └─ glib 2.66.5 *_2, which can be installed;
│  ├─ libglib 2.66.6 would require
│  │  └─ glib 2.66.6 *_2, which can be installed;
│  ├─ libglib 2.66.6 would require
│  │  └─ glib 2.66.6 *_3, which can be installed;
│  ├─ libglib 2.66.7 would require
│  │  └─ glib 2.66.7 *_0, which can be installed;
│  ├─ libglib 2.66.7 would require
│  │  └─ glib 2.66.7 *_1, which can be installed;
│  ├─ libglib 2.68.0 would require
│  │  └─ glib 2.68.0 *_1, which can be installed;
│  ├─ libglib 2.68.0 would require
│  │  └─ glib 2.68.0 *_2, which can be installed;
│  ├─ libglib 2.68.1 would require
│  │  └─ glib 2.68.1 *_0, which can be installed;
│  ├─ libglib 2.68.2 would require
│  │  └─ glib 2.68.2 *_0, which can be installed;
│  ├─ libglib 2.68.2 would require
│  │  └─ glib 2.68.2 *_1, which can be installed;
│  ├─ libglib 2.68.2 would require
│  │  └─ glib 2.68.2 *_2, which can be installed;
│  ├─ libglib 2.68.3 would require
│  │  └─ glib 2.68.3 *_0, which can be installed;
│  └─ libglib 2.68.4 would require
│     └─ glib 2.68.4 *_0, which can be installed;
├─ python 3.9  is not installable because there are no viable options
│  ├─ python 3.9.0 would require
│  │  ├─ libffi >=3.2.1,<3.3.0a0 , which conflicts with any installable versions previously reported;
│  │  └─ openssl >=1.1.1h,<1.1.2a , which can be installed;
│  ├─ python 3.9.0 would require
│  │  ├─ libffi >=3.3,<3.4.0a0 , which conflicts with any installable versions previously reported;
│  │  └─ openssl >=1.1.1h,<1.1.2a , which can be installed;
│  └─ python 3.9.0 conflicts with any installable versions previously reported;
├─ python >=3.8,<4 *cpython* is installable with the potential options
│  ├─ python [3.10.0|3.10.1|...|3.9.9], which can be installed;
│  └─ python 3.12.0rc3 would require
│     └─ _python_rc, which does not exist (perhaps a missing channel);
└─ qt-main is installable with the potential options
   ├─ qt-main 5.15.2 would require
   │  └─ openssl >=3.0.10,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main [5.15.2|5.15.3|5.15.4|5.15.6] would require
   │  └─ libglib [>=2.70.2,<3.0a0 |>=2.72.1,<3.0a0 |>=2.74.1,<3.0a0 ], which can be installed (as previously explained);
   ├─ qt-main 5.15.6 would require
   │  └─ openssl >=3.0.7,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.1.0,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.0.8,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.1.1,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.1.3,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.3.1,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.8 would require
   │  └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported;
   ├─ qt-main 5.15.2 would require
   │  └─ glib >=2.69.1,<3.0a0  with the potential options
   │     ├─ glib [2.69.1|2.70.0|...|2.80.2] would require
   │     │  ├─ libffi [>=3.4,<3.5 |>=3.4,<4.0a0 ], which can be installed;
   │     │  └─ libglib [2.70.0 h174f98d_0|2.70.0 h174f98d_1|...|2.80.0 hf2295e7_4], which can be installed (as previously explained);
   │     └─ glib 2.69.1 conflicts with any installable versions previously reported;
   └─ qt-main 5.15.2 would require
      └─ openssl >=3.0.9,<4.0a0 , which conflicts with any installable versions previously reported.

[JacksonBurns]

@hwpang is the above error being raised by this line? julia -e 'ENV["JULIA_CONDAPKG_BACKEND"] = "Current"; using Pkg; Pkg.add(Pkg.PackageSpec(name="ReactionMechanismSimulator", url="https://github.com/hwpang/ReactionMechanismSimulator.jl.git", rev="fix_installation")); using ReactionMechanismSimulator'

[JacksonBurns]

I ask because I am not sure why we are ever attempting to reinstall python

[hwpang]

@JacksonBurns Yeah, I think so, the error is from those lines when we try to install RMS. I see that there are two specification related to python, one "python==3.9" and the other "conda-forge::python[version='>=3.8,<4',build=cpython]". I think "python==3.9" is just from the existing python in the environment.

Within RMS, it requires PythonCall to be installed, which is where python version = ">=3.8,<4" coming from.

My understanding is that if python already exists in the conda env, both of these commands should do nothing and shouldn't give any error. I am quite confused why requiring to install python again would raise these incompatible errors.

[JacksonBurns]

@hwpang I ran this experiment:

• I built the rmg_env from the environment.yml on this branch
• I installed juliaup using the official sh instructions (which mysteriously still installed Julia up as a conda package in my rmg_env)
• I ran julia -e 'ENV["JULIA_CONDAPKG_BACKEND"] = "Current"; using Pkg; Pkg.add("PythonCall")' and it was successful
• delete .julia, .juliaup and juliaup (tried to just replace julia at first, totally broke and said it couldn't find conda anymore) and reinstall, this time from conda because julialang.org timed out when i tried the shell based install
• ran the RMS installation command above, and again got ERROR: LoadError: InitError: could not find a conda, mamba or micromamba executable

at this point I gave up so I can go catch the train

Can you try and reproduce this locally? I will also try again later.

[JacksonBurns]

Was able to run this morning and got this: ERROR: LoadError: InitError: Python: AttributeError: module 'matplotlib.cm' has no attribute 'get_cmap' .. going to just restart the CI and see what happens.

[rwest]

Brief googling suggests maybe this command is deprecated. "If you want to use a more recent version of matplotlib you can use:

import matplotlib matplotlib.colormaps.get_cmap('...')"

[JacksonBurns]

I think this error is coming from when RMS install PythonPlot - @hwpang is it possible to limit the version of matplotlib that RMS ends up installing?

[hwpang]

We can limit the matplotlib version in the CondaPkg.toml file in RMS. What version of matplotlib do we want to limit it to? The only version requirement for PythonPlot.jl is matplotlib = ">=1".

[JacksonBurns]

Arg, so the CI still isn't even successfully installing all the Julia deps, and worse still every platform is giving a different error:

Intel Mac

``` Looking for: ["matplotlib[version='>=1']", 'python==3.9', "conda-forge::python[version='>=3.8,<4',build=*cpython*]"] Could not solve for environment specs The following packages are incompatible ├─ libglib is installable with the potential options │ ├─ libglib [2.68.4|2.70.0|...|2.80.2] would require │ │ └─ libffi [>=3.4,<4.0a0 |>=3.4.2,<3.5.0a0 ], which can be installed; │ ├─ libglib 2.64.6 would require │ │ └─ glib 2.64.6 *_0, which can be installed; │ ├─ libglib 2.66.1 would require │ │ └─ glib 2.66.1 *_1, which can be installed; │ ├─ libglib 2.66.2 would require │ │ └─ glib 2.66.2 *_0, which can be installed; │ ├─ libglib 2.66.3 would require │ │ └─ glib 2.66.3 *_0, which can be installed; │ ├─ libglib 2.66.3 would require │ │ └─ glib 2.66.3 *_1, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_1, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_2, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_0, which can be installed; │ ├─ libglib 2.66.5 would require │ │ └─ glib 2.66.5 *_2, which can be installed; │ ├─ libglib 2.66.6 would require │ │ └─ glib 2.66.6 *_2, which can be installed; │ ├─ libglib 2.66.6 would require │ │ └─ glib 2.66.6 *_3, which can be installed; │ ├─ libglib 2.66.7 would require │ │ └─ glib 2.66.7 *_0, which can be installed; │ ├─ libglib 2.66.7 would require │ │ └─ glib 2.66.7 *_1, which can be installed; │ ├─ libglib 2.68.0 would require │ │ └─ glib 2.68.0 *_1, which can be installed; │ ├─ libglib 2.68.0 would require │ │ └─ glib 2.68.0 *_2, which can be installed; │ ├─ libglib 2.68.1 would require │ │ └─ glib 2.68.1 *_0, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_0, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_1, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_2, which can be installed; │ ├─ libglib 2.68.3 would require │ │ └─ glib 2.68.3 *_0, which can be installed; │ └─ libglib 2.68.4 would require │ └─ glib 2.68.4 *_0, which can be installed; ├─ python 3.9 is not installable because there are no viable options │ ├─ python 3.9.0 would require │ │ ├─ libffi >=3.2.1,<3.3.0a0 , which conflicts with any installable versions previously reported; │ │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; │ ├─ python 3.9.0 would require │ │ ├─ libffi >=3.3,<3.4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; │ └─ python 3.9.0 conflicts with any installable versions previously reported; ├─ python >=3.8,<4 *cpython* is installable with the potential options │ ├─ python [3.10.0|3.10.1|...|3.9.9], which can be installed; │ └─ python 3.12.0rc3 would require │ └─ _python_rc, which does not exist (perhaps a missing channel); └─ qt-main is installable with the potential options ├─ qt-main 5.15.2 would require │ └─ openssl >=3.0.10,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main [5.15.2|5.15.3|5.15.4|5.15.6|5.15.8] would require │ └─ libglib [>=2.70.2,<3.0a0 |>=2.72.1,<3.0a0 |>=2.74.1,<3.0a0 ], which can be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ libpq >=15.2,<16.0a0 but there are no viable options │ ├─ libpq 15.2 would require │ │ └─ openssl >=3.0.8,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.3 would require │ │ └─ openssl >=3.1.1,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.3 would require │ │ └─ openssl >=3.1.0,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.4 would require │ │ └─ openssl >=3.1.2,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq [15.4|16.0] would require │ │ └─ openssl >=3.1.3,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.4 would require │ │ └─ openssl >=3.2.0,<3.2.0a0 , which does not exist (perhaps a missing channel); │ ├─ libpq [15.4|15.5|16.1] would require │ │ └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq [15.6|16.2] would require │ │ └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ libpq [15.7|16.3] would require │ └─ openssl >=3.3.0,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ libpq >=15.3,<16.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ mysql-libs >=8.0.33,<8.1.0a0 but there are no viable options │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.1,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.3,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ mysql-libs 8.0.33 would require │ └─ openssl >=3.1.4,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ libpq >=16.1,<17.0a0 but there are no viable options │ ├─ libpq [15.4|15.5|16.1], which cannot be installed (as previously explained); │ ├─ libpq [15.6|16.2], which cannot be installed (as previously explained); │ ├─ libpq [15.7|16.3], which cannot be installed (as previously explained); │ └─ libpq 16.1 would require │ └─ openssl >=3.1.4,<3.2.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ libpq >=16.0,<17.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ ├─ libpq >=16.2,<17.0a0 , which cannot be installed (as previously explained); │ └─ mysql-libs >=8.3.0,<8.4.0a0 , which requires │ └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ ├─ krb5 >=1.21.2,<1.22.0a0 but there are no viable options │ │ ├─ krb5 1.21.2 would require │ │ │ └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ krb5 1.21.3 would require │ │ └─ openssl >=3.3.1,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ libpq >=15.4,<16.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ libpq >=16.3,<17.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.2 would require │ └─ openssl >=3.0.9,<4.0a0 , which conflicts with any installable versions previously reported; └─ qt-main 5.15.2 would require └─ glib >=2.69.1,<3.0a0 with the potential options ├─ glib [2.69.1|2.70.0|...|2.80.2] would require │ ├─ libffi >=3.4,<4.0a0 , which can be installed; │ └─ libglib [2.70.0 hf1fb8c0_0|2.70.0 hf1fb8c0_1|...|2.80.0 h81c1438_4], which can be installed (as previously explained); └─ glib 2.69.1 conflicts with any installable versions previously reported. ERROR: LoadError: failed process: Process(`/Users/runner/miniconda3/condabin/mamba install -y -p /Users/runner/miniconda3/envs/rmg_env --override-channels --no-channel-priority "matplotlib[version='>=1']" "python[version='3.9']" "python[version='>=3.8,<4',channel='conda-forge',build='*cpython*']" -c cantera -c conda-forge -c defaults -c mjohnson541 -c rmg`, ProcessExited(1)) [1] ```

M1 Mac

``` Looking for: ["matplotlib[version='>=1']", "conda-forge::python[version='>=3.8,<4',build=*cpython*]", 'python==3.9'] Could not solve for environment specs The following packages are incompatible ├─ python 3.9 is installable with the potential options │ ├─ python 3.9.0 would require │ │ ├─ libffi >=3.2.1,<3.3.0a0 , which can be installed; │ │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; │ └─ python 3.9.0 would require │ ├─ libffi >=3.3,<3.4.0a0 , which can be installed; │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; └─ qt-main is not installable because there are no viable options ├─ qt-main 5.15.2 would require │ └─ openssl >=3.0.10,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main [5.15.2|5.15.3|5.15.4|5.15.6|5.15.8] would require │ ├─ glib >=2.69.1,<3.0a0 , which requires │ │ ├─ libffi >=3.4,<4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ libglib [2.70.0 h67e64d8_0|2.70.0 h67e64d8_1|...|2.80.0 hfc324ee_4], which requires │ │ └─ libffi [>=3.4,<4.0a0 |>=3.4.2,<3.5.0a0 ], which conflicts with any installable versions previously reported; │ └─ libglib [>=2.70.2,<3.0a0 |>=2.72.1,<3.0a0 |>=2.74.1,<3.0a0 ], which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ ├─ libpq >=16.2,<17.0a0 but there are no viable options │ │ ├─ libpq [15.6|16.2] would require │ │ │ └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ libpq [15.7|16.3] would require │ │ └─ openssl >=3.3.0,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ mysql-libs >=8.3.0,<8.4.0a0 , which requires │ └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ libpq >=16.1,<17.0a0 but there are no viable options │ ├─ libpq [15.4|15.5|16.1] would require │ │ └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq [15.6|16.2], which cannot be installed (as previously explained); │ ├─ libpq [15.7|16.3], which cannot be installed (as previously explained); │ └─ libpq 16.1 would require │ └─ openssl >=3.1.4,<3.2.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ ├─ krb5 >=1.21.2,<1.22.0a0 but there are no viable options │ │ ├─ krb5 1.21.2 would require │ │ │ └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ krb5 1.21.3 would require │ │ └─ openssl >=3.3.1,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ libpq >=15.4,<16.0a0 but there are no viable options │ ├─ libpq 15.4 would require │ │ └─ openssl >=3.2.0,<3.2.0a0 , which does not exist (perhaps a missing channel); │ ├─ libpq [15.4|15.5|16.1], which cannot be installed (as previously explained); │ ├─ libpq 15.4 would require │ │ └─ openssl >=3.1.2,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq [15.4|16.0] would require │ │ └─ openssl >=3.1.3,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq [15.6|16.2], which cannot be installed (as previously explained); │ └─ libpq [15.7|16.3], which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ libpq >=16.0,<17.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ libpq >=15.2,<16.0a0 but there are no viable options │ ├─ libpq 15.2 would require │ │ └─ openssl >=3.0.8,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.3 would require │ │ └─ openssl >=3.1.0,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.3 would require │ │ └─ openssl >=3.1.1,<3.2.0a0 , which conflicts with any installable versions previously reported; │ ├─ libpq 15.4, which cannot be installed (as previously explained); │ ├─ libpq [15.4|15.5|16.1], which cannot be installed (as previously explained); │ ├─ libpq 15.4, which cannot be installed (as previously explained); │ ├─ libpq [15.4|16.0], which cannot be installed (as previously explained); │ ├─ libpq [15.6|16.2], which cannot be installed (as previously explained); │ └─ libpq [15.7|16.3], which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ mysql-libs >=8.0.33,<8.1.0a0 but there are no viable options │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.1,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported; │ ├─ mysql-libs 8.0.33 would require │ │ └─ openssl >=3.1.3,<4.0a0 , which conflicts with any installable versions previously reported; │ └─ mysql-libs 8.0.33 would require │ └─ openssl >=3.1.4,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ libpq >=15.3,<16.0a0 , which cannot be installed (as previously explained); ├─ qt-main 5.15.8 would require │ └─ libpq >=16.3,<17.0a0 , which cannot be installed (as previously explained); └─ qt-main 5.15.2 would require └─ openssl >=3.0.9,<4.0a0 , which conflicts with any installable versions previously reported. ERROR: LoadError: failed process: Process(`/Users/runner/miniconda3/condabin/mamba install -y -p /Users/runner/miniconda3/envs/rmg_env --override-channels --no-channel-priority "matplotlib[version='>=1']" "python[version='>=3.8,<4',channel='conda-forge',build='*cpython*']" "python[version='3.9']" -c cantera -c conda-forge -c defaults -c mjohnson541 -c rmg`, ProcessExited(1)) [1] ```

Linux

``` Looking for: ["libstdcxx-ng[version='>=3.4,<13.0']", "matplotlib[version='>=1']", 'python==3.9', "conda-forge::python[version='>=3.8,<4',build=*cpython*]"] Could not solve for environment specs The following packages are incompatible ├─ libglib is installable with the potential options │ ├─ libglib [2.68.4|2.70.0|...|2.80.2] would require │ │ └─ libffi [>=3.4,<4.0a0 |>=3.4.2,<3.5.0a0 ], which can be installed; │ ├─ libglib 2.64.6 would require │ │ └─ glib 2.64.6 *_0, which can be installed; │ ├─ libglib 2.66.1 would require │ │ └─ glib 2.66.1 *_1, which can be installed; │ ├─ libglib 2.66.2 would require │ │ └─ glib 2.66.2 *_0, which can be installed; │ ├─ libglib 2.66.3 would require │ │ └─ glib 2.66.3 *_1, which can be installed; │ ├─ libglib 2.66.3 would require │ │ └─ glib 2.66.3 *_0, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_0, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_1, which can be installed; │ ├─ libglib 2.66.4 would require │ │ └─ glib 2.66.4 *_2, which can be installed; │ ├─ libglib 2.66.5 would require │ │ └─ glib 2.66.5 *_2, which can be installed; │ ├─ libglib 2.66.6 would require │ │ └─ glib 2.66.6 *_2, which can be installed; │ ├─ libglib 2.66.6 would require │ │ └─ glib 2.66.6 *_3, which can be installed; │ ├─ libglib 2.66.7 would require │ │ └─ glib 2.66.7 *_0, which can be installed; │ ├─ libglib 2.66.7 would require │ │ └─ glib 2.66.7 *_1, which can be installed; │ ├─ libglib 2.68.0 would require │ │ └─ glib 2.68.0 *_1, which can be installed; │ ├─ libglib 2.68.0 would require │ │ └─ glib 2.68.0 *_2, which can be installed; │ ├─ libglib 2.68.1 would require │ │ └─ glib 2.68.1 *_0, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_0, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_1, which can be installed; │ ├─ libglib 2.68.2 would require │ │ └─ glib 2.68.2 *_2, which can be installed; │ ├─ libglib 2.68.3 would require │ │ └─ glib 2.68.3 *_0, which can be installed; │ └─ libglib 2.68.4 would require │ └─ glib 2.68.4 *_0, which can be installed; ├─ python 3.9 is not installable because there are no viable options │ ├─ python 3.9.0 would require │ │ ├─ libffi >=3.2.1,<3.3.0a0 , which conflicts with any installable versions previously reported; │ │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; │ ├─ python 3.9.0 would require │ │ ├─ libffi >=3.3,<3.4.0a0 , which conflicts with any installable versions previously reported; │ │ └─ openssl >=1.1.1h,<1.1.2a , which can be installed; │ └─ python 3.9.0 conflicts with any installable versions previously reported; ├─ python >=3.8,<4 *cpython* is installable with the potential options │ ├─ python [3.10.0|3.10.1|...|3.9.9], which can be installed; │ └─ python 3.12.0rc3 would require │ └─ _python_rc, which does not exist (perhaps a missing channel); └─ qt-main is installable with the potential options ├─ qt-main 5.15.2 would require │ └─ openssl >=3.0.10,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main [5.15.2|5.15.3|5.15.4|5.15.6] would require │ └─ libglib [>=2.70.2,<3.0a0 |>=2.72.1,<3.0a0 |>=2.74.1,<3.0a0 ], which can be installed (as previously explained); ├─ qt-main 5.15.6 would require │ └─ openssl >=3.0.7,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.1.0,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.2.1,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.0.8,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.1.1,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.1.3,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.3.1,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.8 would require │ └─ openssl >=3.1.2,<4.0a0 , which conflicts with any installable versions previously reported; ├─ qt-main 5.15.2 would require │ └─ glib >=2.69.1,<3.0a0 with the potential options │ ├─ glib [2.69.1|2.70.0|...|2.80.2] would require │ │ ├─ libffi [>=3.4,<3.5 |>=3.4,<4.0a0 ], which can be installed; │ │ └─ libglib [2.70.0 h174f98d_0|2.70.0 h174f98d_1|...|2.80.0 hf2295e7_4], which can be installed (as previously explained); │ └─ glib 2.69.1 conflicts with any installable versions previously reported; └─ qt-main 5.15.2 would require └─ openssl >=3.0.9,<4.0a0 , which conflicts with any installable versions previously reported. ERROR: LoadError: failed process: Process(`/home/runner/miniconda3/condabin/mamba install -y -p /home/runner/miniconda3/envs/rmg_env --override-channels --no-channel-priority "libstdcxx-ng[version='>=3.4,<13.0']" "matplotlib[version='>=1']" "python[version='3.9']" "python[version='>=3.8,<4',channel='conda-forge',build='*cpython*']" -c cantera -c conda-forge -c defaults -c mjohnson541 -c rmg`, ProcessExited(1)) [1] ```

PythonCall just might not actually be compatible with Python 3.9 (in our specific environment?)? Their CI doesn't actually test again 3.9 specifically, and our package environment is quite complicated.

[JacksonBurns]

@hwpang the function richard shows was removed in 3.9.0 (see the matplotlib release notes) so just less than that I think. But above issue still stands

[JacksonBurns]

I've made a new branch and triggered the CI to see if Python 3.10 will run: https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9761847757

[JacksonBurns]

First Python 3.10 run failed because ringdecomposerlib doesn't support Python 3.10 yet, temporarily commented it out and now running again here: https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9761978305

The maintainer for the rdl feedstock has graciously agreed to add support for our upcoming versions of Python in this PR: https://github.com/conda-forge/ringdecomposerlib-feedstock/pull/4

[hwpang]

Ok I tested the installation locally. I first make an environment using the environment.yml, and tried installing the packages shown in the error message one by one.

This line:

mamba install -y --override-channels --no-channel-priority "python[version='3.9']" -c cantera -c conda-forge -c defaults -c mjohnson541 -c rmg

fails with:

Could not solve for environment specs
The following packages are incompatible
└─ python 3.9  is installable with the potential options
   ├─ python 3.9.0 would require
   │  ├─ libffi >=3.3,<3.4.0a0 , which can be installed;
   │  └─ openssl >=1.1.1h,<1.1.2a , which can be installed;
   └─ python 3.9.0 would require
      ├─ libffi >=3.2.1,<3.3.0a0 , which can be installed;
      └─ openssl >=1.1.1h,<1.1.2a , which can be installed.

I am quite confused by this error message, because the environment comes with python 3.9, and I would have expected it to say that the requirement has been installed.

If I only specify:

mamba install python

it gives the message I would have expected:

Looking for: ['python']

conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache
Transaction

  Prefix: /home/hwpang/miniforge3/envs/rmg_py39

  All requested packages already installed

But for some reason with the version specification:

mamba install "python[version='3.9']" 

it would complain:

Looking for: ['python==3.9']

conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache
Could not solve for environment specs
The following package could not be installed
└─ python 3.9  is installable and it requires
   └─ openssl >=1.1.1h,<1.1.2a , which can be installed.

Why is it not able to tell that python 3.9 is already installed?

Edit:

It seems that it requires * to the version number:

mamba install "python[version='3.9*']"

this will give what I expected:

Looking for: ['python=3.9']

anaconda/linux-64                                           Using cache
anaconda/noarch                                             Using cache
conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache
Transaction

  Prefix: /home/hwpang/miniforge3/envs/rmg_py39

  All requested packages already installed

[hwpang]

I'm running the test after relaxing the python version requirement in RMS from version="3.9" to version=">=3.9" to see if this was the problem. I wasn't aware that version="3.9" doesn't imply version="3.9*".

[hwpang]

In the meantime, I will work on switching the installation of juliaup from the conda binary to the offical suggestion as suggested here.

[rwest]

"The most recent version of matplotlib (v3.9.0) released on May 16 is incompatible with PythonPlot.jl." https://github.com/JuliaPy/PythonPlot.jl/issues/39

[hwpang]

Arg, so the CI still isn't even successfully installing all the Julia deps, and worse still every platform is giving a different error:

Intel Mac M1 Mac Linux PythonCall just might not actually be compatible with Python 3.9 (in our specific environment?)? Their CI doesn't actually test again 3.9 specifically, and our package environment is quite complicated.

@JacksonBurns I've fixed these errors related to python 3.9. And it seems that all that's left is the matplotlib error. Documentation build succeeded

[JacksonBurns]

I'm running the test after relaxing the python version requirement in RMS from version="3.9" to version=">=3.9" to see if this was the problem. I wasn't aware that version="3.9" doesn't imply version="3.9*".

Ah yeah this is a quirk of conda I think

The latest CI has an interessting array of failures:

• Intel Mac is timing out during the Julia install (like it always did)
• M1 Mac is trying to install RMG-molecule for some reason ?? and failing because there are no M1 binaries for that package, see https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9764305750/job/26952342727?pr=2687#step:11:1113 worth nothing too that the Linux build isn't doing this: https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9764305750/job/26952342727?pr=2687#step:11:1113
• Linux is seg faulting during the tests, specifically while running 'test_find_error_canceling_reaction'. This doesn't directly touch any of the code I have changed, but does use lpsolve, which is:
• lp_solve 5.5.2.11 hd590300_0 conda-forge lpsolve55 5.5 py37hda87dfa_1005 conda-forge on main but
• lp_solve 5.5.2.11 hd590300_0 conda-forge lpsolve55 5.5 py39h44dd56e_1007 conda-forge on this branch.

Going to push a commit to just try and get a better stacktrace and maybe some output during the failure

[hwpang]

I'm running the test after relaxing the python version requirement in RMS from version="3.9" to version=">=3.9" to see if this was the problem. I wasn't aware that version="3.9" doesn't imply version="3.9*".

Ah yeah this is a quirk of conda I think

The latest CI has an interessting array of failures:

• Intel Mac is timing out during the Julia install (like it always did)
• M1 Mac is trying to install RMG-molecule for some reason ?? and failing because there are no M1 binaries for that package, see https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9764305750/job/26952342727?pr=2687#step:11:1113 worth nothing too that the Linux build isn't doing this: https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9764305750/job/26952342727?pr=2687#step:11:1113
• Linux is seg faulting during the tests, specifically while running 'test_find_error_canceling_reaction'. This doesn't directly touch any of the code I have changed, but does use lpsolve, which is:
• lp_solve 5.5.2.11 hd590300_0 conda-forge lpsolve55 5.5 py37hda87dfa_1005 conda-forge on main but
• lp_solve 5.5.2.11 hd590300_0 conda-forge lpsolve55 5.5 py39h44dd56e_1007 conda-forge on this branch.

Going to push a commit to just try and get a better stacktrace and maybe some output during the failure

I was able to make the MacOS to not install rmgmolecule. However, now we run into a new error:

ERROR: LoadError: InitError: ReadOnlyMemoryError()
during initialization of module PythonPlot
in expression starting at /Users/runner/miniconda3/envs/rmg_env/.julia/packages/ReactionMechanismSimulator/C6833/src/Plotting.jl:1
in expression starting at /Users/runner/miniconda3/envs/rmg_env/.julia/packages/ReactionMechanismSimulator/C6833/src/ReactionMechanismSimulator.jl:1
Matplotlib is building the font cache; this may take a moment.

[JacksonBurns]

Ok so interesting developments - the Linux build continues to fail when calling out to Lpsolve but the Intel Mac (which didn't use to even install RMS at all, so this is surprising) passes that test, later failing during this unit test because of something I did with Cython, I think:

 >   def same_object(object1, object2, _check_identical, _only_check_label,
                 _generate_initial_map, _strict, _save_order):
E                TypeError: same_object() takes exactly 7 positional arguments (2 given)

rmgpy/reaction.py:1545: TypeError

The M1 error... no idea. Our current docs tell users to just have the M1 hardware emulate the old arch so that they can run old arch binaries - perhaps we do the same in the CI?

[hwpang]

Ok so interesting developments - the Linux build continues to fail when calling out to Lpsolve but the Intel Mac (which didn't use to even install RMS at all, so this is surprising) passes that test, later failing during this unit test because of something I did with Cython, I think:

 >   def same_object(object1, object2, _check_identical, _only_check_label,
                 _generate_initial_map, _strict, _save_order):
E                TypeError: same_object() takes exactly 7 positional arguments (2 given)

rmgpy/reaction.py:1545: TypeError

The M1 error... no idea. Our current docs tell users to just have the M1 hardware emulate the old arch so that they can run old arch binaries - perhaps we do the same in the CI?

Yeah that sounds fine to me.

[JacksonBurns]

@hwpang ARM mac is now installing Intel packages, but this error pops up during RMS installation:

Looking for: ["matplotlib[version='>=1']", "conda-forge::python[version='>=3.8,<4',build=*cpython*]"]

Downloading and Extracting Packages: ...working... done
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Python path configuration:
  PYTHONHOME = '/Users/runner/miniconda3/envs/rmg_env:/Users/runner/miniconda3/envs/rmg_env'
  PYTHONPATH = '/Users/runner/work/RMG-Py/RMG-Py/RMG-Py:'
  program name = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12'
  sys._base_executable = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  sys.base_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.base_exec_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.platlibdir = 'lib'
  sys.executable = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  sys.prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.exec_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.path = [
    '/Users/runner/work/RMG-Py/RMG-Py/RMG-Py',
    '/Users/runner/miniconda3/envs/rmg_env/.julia/packages/ReactionMechanismSimulator/NVrJU/deps',
    '/Users/runner/miniconda3/envs/rmg_env/lib/python312.zip',
    '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12',
    '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

[JacksonBurns]

Stack overflow says it's because we are setting PYTHONPATH https://stackoverflow.com/a/65740883

[hwpang]

Stack overflow says it's because we are setting PYTHONPATH https://stackoverflow.com/a/65740883

Thanks. The only PYTHONPATH it has is the PYTHONPATH we set for the RMG-Py, and we didn't set any other PYTHONPATH. Any ideas on what we could try?

Python path configuration:
  PYTHONHOME = '/Users/runner/miniconda3/envs/rmg_env:/Users/runner/miniconda3/envs/rmg_env'
  PYTHONPATH = '/Users/runner/work/RMG-Py/RMG-Py/RMG-Py:'

[hwpang]

So unsetting the PYTHONPATH didn't solve the problem we encountered with ARM mac. It failed with the same error:

Downloading and Extracting Packages: ...working... done
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Python path configuration:
  PYTHONHOME = '/Users/runner/miniconda3/envs/rmg_env:/Users/runner/miniconda3/envs/rmg_env'
  PYTHONPATH = (not set)
  program name = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12'
  sys._base_executable = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  sys.base_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.base_exec_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.platlibdir = 'lib'
  sys.executable = '/Users/runner/miniconda3/envs/rmg_env/bin/python'
  sys.prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.exec_prefix = '/Users/runner/miniconda3/envs/rmg_env'
  sys.path = [
    '/Users/runner/miniconda3/envs/rmg_env/lib/python312.zip',
    '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12',
    '/Users/runner/miniconda3/envs/rmg_env/lib/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000001fe960c00 (most recent call first):
  <no Python frame>
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70
  [2] (::Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String})()
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1157
  [3] withenv(::Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, ::Pair{String, String}, ::Vararg{Pair{String}})
    @ Base ./env.jl:257
  [4] (::Pkg.Operations.var"#117#122"{String, Bool, Bool, Bool, Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.PackageSpec})()
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1825
  [5] with_temp_env(fn::Pkg.Operations.var"#117#122"{String, Bool, Bool, Bool, Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.PackageSpec}, temp_env::String)
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1706
  [6] (::Pkg.Operations.var"#115#120"{Dict{String, Any}, Bool, Bool, Bool, Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String})(tmp::String)
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1[795](https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/9844707391/job/27178685242?pr=2687#step:12:796)
  [7] mktempdir(fn::Pkg.Operations.var"#115#120"{Dict{String, Any}, Bool, Bool, Bool, Pkg.Operations.var"#67#74"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:766
  [8] mktempdir(fn::Function, parent::String)
    @ Base.Filesystem ./file.jl:762
  [9] mktempdir
    @ ./file.jl:762 [inlined]
 [10] sandbox(fn::Function, ctx::Pkg.Types.Context, target::Pkg.Types.PackageSpec, target_path::String, sandbox_path::String, sandbox_project_override::Pkg.Types.Project; preferences::Dict{String, Any}, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1753
 [11] build_versions(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}; verbose::Bool)
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1138
 [12] build_versions
    @ ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1056 [inlined]
 [13] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Set{Base.UUID}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform)
    @ Pkg.Operations ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1399
 [14] add
    @ ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1377 [inlined]
 [15] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::@Kwargs{io::Base.PipeEndpoint})
    @ Pkg.API ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:278
 [16] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.PipeEndpoint, kwargs::@Kwargs{})
    @ Pkg.API ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:159
 [17] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:148
 [18] add(pkg::Pkg.Types.PackageSpec)
    @ Pkg.API ~/miniconda3/envs/rmg_env/.julia/juliaup/julia-1.10.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:146
 [19] top-level scope
    @ none:1
Error: Process completed with exit code 1.

It has also caused the previously working Ubuntu installation to fail. So I'm gonna drop that commit. Would appreciate any ideas to try. @JacksonBurns

[rwest]

This looks promising, right? What @hwpang pushed yesterday compiled and installed everything and started to run all the unit tests? Then it seg-faulted on a line status = lpsolve('solve', lp)? (But maybe that's just where we were last week?)

Update: it could feasibly be the underlying lp_solve package causing issues not just the lpsolve55 which is the wrapper/API.

[JacksonBurns]

Apologies for the delay, travel and also @hwpang is Dr. @hwpang now 🎉!

I am not sure what else to try RE: ARM Mac is busted. Running ARM Packages natively failed at the Julia installation, emulating Intel packages failed during Julia installation also but for a different, possibly PYTHONPATH related reason. I will be back in front of an ARM Mac a week from now, which might make debugging easier, but I would appreciate help here.

Intel Mac is broken on GitHub because it is too slow to install RMS (except when sometimes it does, for some reason, only to fail due to a Cython error I think I caused). Ubuntu is failing because lpsolve crashes the whole test run, and frustratingly refuses to even give a helpful trace.

For now, maybe we try and figure out why lpsolve is failing?

[rwest]

How hard is it to remove the need for LPSolve?.... https://github.com/ReactionMechanismGenerator/RMG-Py/pull/2623 and https://github.com/ReactionMechanismGenerator/RMG-Py/pull/2596#issuecomment-1977986049

[hwpang]

How hard is it to remove the need for LPSolve?.... #2623 and #2596 (comment)

@rwest This is sort of a chicken and egg problem, because replacing lpsolve with scipy.optimize.milp requires python 3.9, and this PR that upgrades RMG to python 3.9 is failing due to lpsolve :(

[hwpang]

An alternative thought, maybe we can rebase https://github.com/ReactionMechanismGenerator/RMG-Py/pull/2623 on this PR and see if that resolves the lpsolve issue?

[rwest]

Worth a try. We have both the chickens and the eggs as open PRs. We should try putting them together.

[JacksonBurns]

I will (1) make a copy of @xiaoruiDong's branch in #2623 and push it to ReactionMechanismGenerator/RMG-Py and then (2) rebase that branch on this branch and then (3) open a PR from that branch into this one

[JacksonBurns]

@hwpang's #2596 is also required in combination with #2694 in order to completely remove lpsolve. I will (1) make a copy of that branch on RMG-Py (2) rebase it onto #2694 (3) tack on a commit to remove lpsolve (4) open a PR into #2694

Merge order would then be Isodesmic into CLAR, CLAR into this, and this into main.

[rwest]

I lost track of the other PRs. Is this idea of removing lpsolve looking promising?

[JacksonBurns]

I lost track of the other PRs. Is this idea of removing lpsolve looking promising?

Yes - the combination of #2694 and #2695 will allow us to remove lpsolve (as well as roll in some bug fixes), which should allow merging this PR.

[JacksonBurns]

Don't fix merge conflicts in this PR, I have already done so in #2694