[CONDA-ENV] Problem when creating sparta environment

[orion160]

+ cd ./scripts/..
+ ./scripts/render_recipe_for_platform.sh rendered_recipe.icXLm0
conda is /home/cc/miniconda3/bin/conda
jq is /home/cc/miniconda3/bin/jq
yq is /home/cc/miniconda3/bin/yq
Rendering recipe in /home/cc/map/conda.recipe
Warning: 'conda-forge' already in 'channels' list, moving to the top
+ conda render -m .ci_support/linux_64_.yaml --file rendered_recipe.icXLm0 conda.recipe
/home/cc/miniconda3/lib/python3.12/site-packages/conda_build/environ.py:544: UserWarning: The environment variable 'SYSTEM_DEFAULTWORKINGDIRECTORY' is being passed through with value '/home/cc/map'.  If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
  warnings.warn(
--------------
Hash contents:
--------------
{'boost': '1.78.0',
 'boost_cpp': '1.78.0',
 'c_compiler': 'gcc',
 'c_compiler_version': '12',
 'channel_targets': 'conda-forge main',
 'cxx_compiler': 'gxx',
 'cxx_compiler_version': '12',
 'hdf5': '1.14.2',
 'linux_clang_version': '15',
 'numpy': '1.21',
 'python': '3.10.* *_cpython',
 'sqlite': '3',
 'target_platform': 'linux-64',
 'zlib': '1.2'}
----------
meta.yaml:
----------
Wrote yaml to rendered_recipe.icXLm0
::SUCCESS:: Rendered recipe written to 'rendered_recipe.icXLm0'
+ ./scripts/rendered2env_yaml.sh rendered_recipe.icXLm0 dev
+ rm rendered_recipe.icXLm0
+ set -x
+ conda env create -f=environment.icXLm0.yml -n sparta

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 18, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 84, in main_subshell
        exit_code = do_call(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 200, in do_call
        result = getattr(module, func_name)(args, parser)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/notices/core.py", line 132, in wrapper
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/cli/main_env_create.py", line 170, in execute
        result[installer_type] = installer.install(
                                 ^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/env/installers/conda.py", line 48, in install
        solver = _solve(prefix, specs, args, env, *_, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/env/installers/conda.py", line 32, in _solve
        solver = solver_backend(prefix, channels, subdirs, specs_to_add=specs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda_libmamba_solver/solver.py", line 87, in __init__
        super().__init__(
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/core/solve.py", line 97, in __init__
        self.specs_to_add = frozenset(MatchSpec.merge(s for s in specs_to_add))
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/models/match_spec.py", line 504, in merge
        reduce(lambda x, y: x._merge(y, union), group)
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/models/match_spec.py", line 504, in <lambda>
        reduce(lambda x, y: x._merge(y, union), group)
                            ^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/models/match_spec.py", line 536, in _merge
        final = this_component.merge(that_component)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cc/miniconda3/lib/python3.12/site-packages/conda/models/match_spec.py", line 828, in merge
        raise ValueError(
    ValueError: Incompatible component merge:
      - 'h4ab18f5_6'
      - 'h4ab18f5_1'

`$ /home/cc/miniconda3/bin/conda env create -f=environment.icXLm0.yml -n sparta`

  environment variables:
                 CIO_TEST=<not set>
    CONDA_ALLOW_SOFTLINKS=false
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/home/cc/miniconda3/bin/conda
             CONDA_PREFIX=/home/cc/miniconda3
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/home/cc/miniconda3/bin/python
               CONDA_ROOT=/home/cc/miniconda3
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
               LD_PRELOAD=<not set>
                     PATH=/home/cc/miniconda3/bin:/home/cc/miniconda3/condabin:/usr/local/sbin:/
                          usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/game
                          s:/snap/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : base
    active env location : /home/cc/miniconda3
            shell level : 1
       user config file : /home/cc/.condarc
 populated config files : /home/cc/.condarc
          conda version : 24.5.0
    conda-build version : 24.5.1
         python version : 3.12.2.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake_avx512
                          __conda=24.5.0=0
                          __glibc=2.35=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /home/cc/miniconda3  (writable)
      conda av data dir : /home/cc/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/cc/miniconda3/pkgs
                          /home/cc/.conda/pkgs
       envs directories : /home/cc/miniconda3/envs
                          /home/cc/.conda/envs
               platform : linux-64
             user-agent : conda/24.5.0 requests/2.32.2 CPython/3.12.2 Linux/5.15.0-113-generic ubuntu/22.04.4 glibc/2.35 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4 c/. s/. e/.
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

An unexpected error has occurred. Conda has prepared the above report.
If you suspect this error is being caused by a malfunctioning plugin,
consider using the --no-plugins option to turn off plugins.

Example: conda --no-plugins install <package>

Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
the command line to run the command without plugins enabled.

Example: CONDA_NO_PLUGINS=true conda install <package>

If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers? [y/N]: 
Timeout reached. No report sent.

[leonmatt]

I am also running into this issue

[klingaard]

Yeah, this was brought to my attention last week. I have a colleague who got it working again; I'll ask him to post a fix.

[colbynyce-mips]

I worked around this issue by tweaking the environment.$$$.yml file directly to remove the clashing duplicates. If you open environment.icXLm0.yml, you should see dups for zlib and libzlib. If you remove the dups so that you are only left with the newer of the two versions, you should then be able to rerun this command and complete the Sparta environment setup:

/home/cc/miniconda3/bin/conda env create -f=environment.icXLm0.yml -n sparta

[klingaard]

Curious, what version of conda did you guys use? I just tied it with conda 24.7.1 with no issues...

[klingaard]

One item to note is the conda feedstock is tied to CI rendered from conda smithy. I have a PR to try to get this working again (and might also fix the conda environment setup). PR #510

[klingaard]

Welp, I slept on it and tried again this morning and have NEW errors. Something's borked for sure. Thinking about supplying a fixed environment file that everyone can use (and update it every now and then).

[klingaard]

I narrowed it down to conda smithy insisting to move us to boost 1.85, which has dependencies on newer libraries which are in conflict with older libraries in the same rendering. I'm trying to get the renderer to include boost 1.78, but it's gleefully ignored -- in fact, BOTH 1.78 and 1.85 are trying to be installed. I have no idea which environment/library is insisting on installing it.

I manually removed boost 1.85 from the rendered environment file and had 100% success in package resolution as well as a clean sparta build. :man_shrugging:

@timsnyder do you know how to force the render to NOT include a specific package?

[klingaard]

For now, I've created a safe environment for render: https://github.com/sparcians/map/blob/master/scripts/rendered_safe_environment.yaml and updated the main README (step 5) to use it.