conda-incubator / conda-store

Data science environments, for collaboration. ✨
https://conda.store
BSD 3-Clause "New" or "Revised" License
144 stars 46 forks source link

[BUG] - `test_solve_lockfile_multiple_platforms[simple_specification]` is flaky on CI #789

Closed peytondmurray closed 6 months ago

peytondmurray commented 6 months ago

Describe the bug

The test_solve_lockfile_multiple_platforms[simple_specification] test is flaky on CI. See https://github.com/conda-incubator/conda-store/actions/runs/8288925479/job/22684452762?pr=776#step:7:70 for an example of this failing on a CI run for a PR that made unrelated changes.

Expected behavior

The test is not flaky.

How to Reproduce the problem?

This test is flaky; check the Actions for this repo and we should be able to see CI failures associated with this.

Output

Run pytest -m "not extended_prefix and not user_journey" tests
============================= test session starts ==============================
platform darwin -- Python 3.10.0, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/runner/work/conda-store/conda-store/conda-store-server
configfile: pyproject.toml
plugins: celery-0.0.0a1, playwright-0.4.4, mock-3.12.0, anyio-3.7.1, base-url-2.1.0
collected 193 items / 5 deselected / 188 selected

tests/test_actions.py .....F..........xx..................               [ 19%]
tests/test_app_api.py .....                                              [ 21%]
tests/test_auth.py ..................................................... [ 50%]
......                                                                   [ 53%]
tests/test_db_api.py .......                                             [ 56%]
tests/test_server.py ................................................... [ 84%]
...................                                                      [ 94%]
tests/test_testing.py .                                                  [ 94%]
tests/test_traitlets.py .........                                        [ 99%]
tests/test_usage.py .                                                    [100%]

=================================== FAILURES ===================================
_________ test_solve_lockfile_multiple_platforms[simple_specification] _________

conda_store = <conda_store_server.app.CondaStore object at 0x111f0cbe0>
specification = CondaSpecification(name='test', channels=['main'], dependencies=['zlib'], variables=None, prefix=None, description='')
request = <FixtureRequest for <Function test_solve_lockfile_multiple_platforms[simple_specification]>>

    @pytest.mark.parametrize(
        "specification",
        [
            "simple_specification",
            "simple_specification_with_pip",
        ],
    )
    def test_solve_lockfile_multiple_platforms(conda_store, specification, request):
        specification = request.getfixturevalue(specification)
>       context = action.action_solve_lockfile(
            conda_command=conda_store.conda_command,
            specification=specification,
            platforms=["osx-64", "linux-64", "win-64", "osx-arm64"],
        )

/Users/runner/work/conda-store/conda-store/conda-store-server/tests/test_actions.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/runner/work/conda-store/conda-store/conda-store-server/conda_store_server/action/base.py:31: in wrapper
    action_context.result = f(action_context, *args, **kwargs)
/Users/runner/work/conda-store/conda-store/conda-store-server/conda_store_server/action/generate_lockfile.py:52: in action_solve_lockfile
    run_lock(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py:1110: in run_lock
    make_lock_files(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py:393: in make_lock_files
    fresh_lock_content = create_lockfile_from_spec(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py:836: in create_lockfile_from_spec
    deps = _solve_for_arch(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py:748: in _solve_for_arch
    conda_deps = solve_conda(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_solver.py:160: in solve_conda
    dry_run_install = solve_specs_for_arch(
/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/conda_solver.py:372: in solve_specs_for_arch
    proc.check_returncode()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = CompletedProcess(args=['/Users/runner/miniconda3/envs/test/bin/mamba', 'create', '--prefix', '/var/folders/h1/8hndypj1...c version not found (virtual package skipped)\nerror    libmamba ZSTD decompression error: Unknown frame descriptor\n")

    def check_returncode(self):
        """Raise CalledProcessError if the exit code is non-zero."""
        if self.returncode:
>           raise CalledProcessError(
                self.returncode, self.args, self.stdout, self.stderr
            )
E           conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command '['/Users/runner/miniconda3/envs/test/bin/mamba', 'create', '--prefix', '/var/folders/h1/8hndypj13nsbj5pn4xsnv1tm0000gn/T/tmp2pem4waa/prefix', '--dry-run', '--json', '--strict-channel-priority', '--override-channels', '--channel', 'main', '--channel', 'file:///Users/runner/work/_temp/tmpo5kggjir', 'zlib']' returned non-zero exit status 1.

/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_lock/_vendor/poetry/utils/_compat.py:168: CalledProcessError
---------------------------- Captured stderr setup -----------------------------
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 48be4072fe58, initial schema
INFO  [alembic.runtime.migration] Running upgrade 48be4072fe58 -> 8d63a091aff8, Add Environment.description
INFO  [alembic.runtime.migration] Running upgrade 8d63a091aff8 -> 5ad723de2abd, Adding CONTAINER_REGISTRY value to enum
INFO  [alembic.runtime.migration] Running upgrade 5ad723de2abd -> 16f65805dc8f, split conda_package into conda_package and conda_package_build
INFO  [alembic.runtime.migration] Running upgrade 16f65805dc8f -> abd7248d5327, adding a settings table
INFO  [alembic.runtime.migration] Running upgrade abd7248d5327 -> b387747ca9b7, role mapping
INFO  [alembic.runtime.migration] Running upgrade b387747ca9b7 -> d78e9889566a, add status_info
INFO  [alembic.runtime.migration] Running upgrade d78e9889566a -> 30b37e725c32, add build_key_version
INFO  [alembic.runtime.migration] Running upgrade 30b37e725c32 -> 771180018e1b, add v2 role mappings
INFO  [alembic.runtime.migration] Running upgrade 771180018e1b -> 0f7e23ff24ee, add worker
INFO  [alembic.runtime.migration] Running upgrade 0f7e23ff24ee -> 57cd11b949d5, add installer
INFO  [alembic.runtime.migration] Running upgrade 57cd11b949d5 -> 03c839888c82, add canceled status
=============================== warnings summary ===============================
../../../../miniconda3/envs/test/lib/python3.10/site-packages/conda_pack/core.py:15
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/conda_pack/core.py:15: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

../../../../miniconda3/envs/test/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
../../../../miniconda3/envs/test/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
../../../../miniconda3/envs/test/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
../../../../miniconda3/envs/test/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/pkg_resources/__init__.py:2832: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

conda_store_server/action/generate_constructor_installer.py:87
  /Users/runner/work/conda-store/conda-store/conda-store-server/conda_store_server/action/generate_constructor_installer.py:87: DeprecationWarning: invalid escape sequence '\S'
    post_install = """\

../../../../miniconda3/envs/test/lib/python3.10/site-packages/traitlets/traitlets.py:3615
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/traitlets/traitlets.py:3615: DeprecationWarning: metadata {'description': 'Conda platforms to solve environments for via conda-lock. Must include current platform.'} was set from the constructor. With traitlets 4.1, metadata should be set using the .tag() method, e.g., Int().tag(key1='value1', key2='value2')
    super().__init__(trait=trait, default_value=default_value, **kwargs)

tests/test_server.py:419
  /Users/runner/work/conda-store/conda-store/conda-store-server/tests/test_server.py:419: PytestUnknownMarkWarning: Unknown pytest.mark.extended_prefix - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.extended_prefix

tests/test_actions.py::test_solve_lockfile[simple_specification]
  /Users/runner/work/conda-store/conda-store/conda-store-server/conda_store_server/build.py:110: DeprecationWarning: invalid escape sequence '\d'
    match = re.fullmatch("build-(\d+)-(.*)", str(task["id"]))

tests/test_actions.py::test_add_conda_prefix_packages[conda_prefix0]
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/kombu/transport/sqlalchemy/models.py:63: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings.  Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
    self.queue = queue

tests/test_app_api.py: 1 warning
tests/test_server.py: 10 warnings
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
  whether broker connection retries are made during startup in Celery 6.0 and above.
  If you wish to retain the existing behavior for retrying connections on startup,
  you should set broker_connection_retry_on_startup to True.
    warnings.warn(

tests/test_auth.py: 16 warnings
tests/test_server.py: 70 warnings
tests/test_traitlets.py: 13 warnings
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead.
    warnings.warn(message, DeprecationWarning)

tests/test_traitlets.py::test_conda_store_server_enable_ui
tests/test_traitlets.py::test_conda_store_server_url_prefix
tests/test_traitlets.py::test_conda_store_server_url_prefix
tests/test_traitlets.py::test_conda_store_server_templates
  /Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/starlette/templating.py:178: DeprecationWarning: The `name` is not the first parameter anymore. The first parameter should be the `Request` instance.
  Replace `TemplateResponse(name, {"request": request})` by `TemplateResponse(request, name)`.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_actions.py::test_solve_lockfile_multiple_platforms[simple_specification] - conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command '['/Users/runner/miniconda3/envs/test/bin/mamba', 'create', '--prefix', '/var/folders/h1/8hndypj13nsbj5pn4xsnv1tm0000gn/T/tmp2pem4waa/prefix', '--dry-run', '--json', '--strict-channel-priority', '--override-channels', '--channel', 'main', '--channel', 'file:///Users/runner/work/_temp/tmpo5kggjir', 'zlib']' returned non-zero exit status 1.
= 1 failed, 185 passed, 5 deselected, 2 xfailed, 124 warnings in 887.36s (0:14:47) =
Error: Process completed with exit code 1.

Versions and dependencies used.

No response

Anything else?

No response

peytondmurray commented 6 months ago

Duplicate of https://github.com/conda-incubator/conda-store/issues/783. Closing.