search

openforcefield / openff-toolkit

The Open Forcefield Toolkit provides implementations of the SMIRNOFF format, parameterization engine, and other tools. Documentation available at http://open-forcefield-toolkit.readthedocs.io
http://openforcefield.org
MIT License
318 stars 93 forks source link

A freesolv GBSA energy test is failing on mac-latest CI #1964

Closed j-wags closed 1 week ago

j-wags commented 2 weeks ago

Describe the bug

=================================== FAILURES ===================================
_ TestForceFieldParameterAssignment.test_freesolv_gbsa_energies[2523689-0_0_2-False-OBC2-True] _
[gw1] darwin -- Python 3.10.15 /Users/runner/micromamba/envs/openff-toolkit-test/bin/python
openff/toolkit/_tests/test_forcefield.py:3472: in test_freesolv_gbsa_energies
    compare_system_energies(
openff/toolkit/_tests/utils.py:676: in compare_system_energies
    potential_energy1, potential_energy2 = compare_context_energies(
openff/toolkit/_tests/utils.py:504: in compare_context_energies
    raise_assert(
openff/toolkit/_tests/utils.py:497: in raise_assert
    raise FailedEnergyComparisonError(
E   openff.toolkit._tests.utils.FailedEnergyComparisonError: potential energy 1 -0.9687832195585031 kJ/mol, potential energy 2: -0.9686264585172353 kJ/mol

https://github.com/openforcefield/openff-toolkit/actions/runs/11788161797/job/32866381468#step:15:371

Unable to reproduce locally, but what I've tried is:

micromamba env create -n off-tk-dev-arm -f devtools/conda-envs/test_env.yaml python=3.10
micromamba activate off-tk-dev-arm
pip install -e .
for i in {1..30};do;pytest -vv openff/toolkit/_tests/test_forcefield.py -k "test_freesolv_gbsa_energies[2523689-0_0_2-False-OBC2-True]" --runslow | grep "2523689-0_0_2-False-OBC2-True";done

yields all passes.

local and test envs are nearly identical, only difference seems to be tomli:

(off-tk-dev-arm) jw@mba$ difft xx_local_arm.tsv xx_github_arm.tsv                                                                                                                      
xx_github_arm.tsv --- 1/2 --- Text
1 List of packages in environment: "/Users/jeffreywagner/micromamba/envs/off-tk-dev-arm"           1 List of packages in environment: "/Users/runner/micromamba/envs/openff-toolkit-test"
2                                                                                                  2 
3   Name                           Version         Build                              Channel      3   Name                           Version         Build                              Channel    
4 ──────────────────────────────────────────────────────────────────────────────────────────────── 4 ────────────────────────────────────────────────────────────────────────────────────────────────
  ─                                                                                                  ─

xx_github_arm.tsv --- 2/2 --- Text
277   tinycss2                       1.4.0           pyhd8ed1ab_0                       conda-forg 277   tinycss2                       1.4.0           pyhd8ed1ab_0                       conda-forg
... e                                                                                              ... e
278   tk                             8.6.13          h5083fa2_1                         conda-forg 278   tk                             8.6.13          h5083fa2_1                         conda-forg
... e                                                                                              ... e
279   toml                           0.10.2          pyhd8ed1ab_0                       conda-forg 279   toml                           0.10.2          pyhd8ed1ab_0                       conda-forg
... e                                                                                              ... e
280   tomli                          2.1.0           pyhff2d567_0                       conda-forg 280   tomli                          2.0.2           pyhd8ed1ab_0                       conda-forg
... e                                                                                              ... e
281   torchmetrics                   1.5.2           pyhe5570ce_0                       conda-forg 281   torchmetrics                   1.5.2           pyhe5570ce_0                       conda-forg
... e                                                                                              ... e
282   tornado                        6.4.1           py310h493c2e1_1                    conda-forg 282   tornado                        6.4.1           py310h493c2e1_1                    conda-forg
... e                                                                                              ... e
283   tqdm                           4.67.0          pyhd8ed1ab_0                       conda-forg 283   tqdm                           4.67.0          pyhd8ed1ab_0                       conda-forg
    e                                                                                                  e