search

conda-incubator / conda-store

Data science environments, for collaboration. ✨
https://conda.store
BSD 3-Clause "New" or "Revised" License
143 stars 46 forks source link

[BUG] - Flaky CI tests: libmamba.so.2: undefined symbol: solver_ruleinfo2str, version SOLV_1.0 #632

Closed nkaretnikov closed 11 months ago

nkaretnikov commented 11 months ago

Describe the bug

Our integration tests started failing very recently with this. Seems to be flaky, too:

2023-10-17T19:54:25.7224823Z ##[warning]    import libmambapy as api
  File "/usr/share/miniconda/lib/python3.11/site-packages/libmambapy/__init__.py", line 7, in <module>
    raise e
  File "/usr/share/miniconda/lib/python3.11/site-packages/libmambapy/__init__.py", line 4, in <module>
    from libmambapy.bindings import *  # noqa: F401,F403
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: /usr/share/miniconda/lib/python3.11/site-packages/libmambapy/../../../libmamba.so.2: undefined symbol: solver_ruleinfo2str, version SOLV_1.0

2023-10-17T19:54:25.7228829Z     import libmambapy as api
2023-10-17T19:54:25.7229823Z   File "/usr/share/miniconda/lib/python3.11/site-packages/libmambapy/__init__.py", line 7, in <module>
2023-10-17T19:54:25.7230742Z     raise e
2023-10-17T19:54:25.7231614Z   File "/usr/share/miniconda/lib/python3.11/site-packages/libmambapy/__init__.py", line 4, in <module>
2023-10-17T19:54:25.7232658Z     from libmambapy.bindings import *  # noqa: F401,F403
2023-10-17T19:54:25.7233240Z     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-17T19:54:25.7234655Z ImportError: /usr/share/miniconda/lib/python3.11/site-packages/libmambapy/../../../libmamba.so.2: undefined symbol: solver_ruleinfo2str, version SOLV_1.0

This blocks merges because tests are always failing.

I'm not sure yet what has changed. Some new dependencies were added to environment-dev.yaml. Also, there were changes to docker-compose.yaml. Or maybe it's something else.

Expected behavior

Tests do not fail for no reason.

How to Reproduce the problem?

Seen this on the integration-test conda-store-server job on CI.

Output

No response

Versions and dependencies used.

01ef8962f48bbfa21197301619511aa0857c8e76

Anything else?

No response

nkaretnikov commented 11 months ago

For example: https://github.com/conda-incubator/conda-store/actions/runs/6551631950/job/17793244879?pr=625 https://github.com/conda-incubator/conda-store/actions/runs/6549296545/job/17797855369?pr=607 (these are different PRs)

nkaretnikov commented 11 months ago

I think this might be related: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20231001.1 The ubuntu image was updated 2 weeks ago. Since we use ubuntu-latest, we also automatically use it. I'll try rolling back to the previous image.

nkaretnikov commented 11 months ago

It'd be nice to pin OS versions used in GH actions regardless of whether it's related to this issue or not, also on macOS.

nkaretnikov commented 11 months ago

I'm trying various images here https://github.com/conda-incubator/conda-store/pull/607

nkaretnikov commented 11 months ago

Ubuntu LTS releases: https://wiki.ubuntu.com/Releases