conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
152 stars 181 forks source link

Potential regression for handling pins #1615

Open kmod opened 2 years ago

kmod commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

mpy4py-feedstock is currently failing to rerender:

$ conda-smithy rerender
INFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2022.03.28.19.38.17
INFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpvk64f_b0
INFO:conda_smithy.configure_feedstock:pypy37-windows.yaml from feedstock is ignored and upstream version is used
INFO:conda_smithy.configure_feedstock:pypy37.yaml from feedstock is ignored and upstream version is used
INFO:conda_smithy.configure_feedstock:python310.yaml from feedstock is ignored and upstream version is used
Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.
WARNING:conda_build.config:Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.
Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you're doing.
WARNING:conda_build.config:Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you're doing.
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Adding in variants from /tmp/tmpvk64f_b0/conda_build_config.yaml
INFO:conda_build.variants:Adding in variants from /tmp/tmpvk64f_b0/conda_build_config.yaml
Adding in variants from /home/kmod/pyston/mpi4py-feedstock/recipe/conda_build_config.yaml
INFO:conda_build.variants:Adding in variants from /home/kmod/pyston/mpi4py-feedstock/recipe/conda_build_config.yaml
INFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpvk64f_b0/share/conda-forge/migrations/pypy37.yaml,/tmp/tmpvk64f_b0/share/conda-forge/migrations/pypy37-windows.yaml,/tmp/tmpvk64f_b0/share/conda-forge/migrations/python310.yaml
Adding in variants from argument_variants
INFO:conda_build.variants:Adding in variants from argument_variants
INFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpvk64f_b0/share/conda-forge/migrations/pypy37.yaml,/tmp/tmpvk64f_b0/share/conda-forge/migrations/pypy37-windows.yaml,/tmp/tmpvk64f_b0/share/conda-forge/migrations/python310.yaml
Traceback (most recent call last):
  File "/home/kmod/miniconda3/bin/conda-smithy", line 10, in <module>
    sys.exit(main())
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/cli.py", line 681, in main
    args.subcommand_func(args)
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/cli.py", line 486, in __call__
    self._call(args, tmpdir)
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/cli.py", line 491, in _call
    configure_feedstock.main(
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 2262, in main
    render_travis(env, config, forge_dir, return_metadata=True)
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 1155, in render_travis
    return _render_ci_provider(
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 758, in _render_ci_provider
    dump_subspace_config_files(
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 431, in dump_subspace_config_files
    configs, top_level_loop_vars = _collapse_subpackage_variants(
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 375, in _collapse_subpackage_variants
    break_up_top_level_values(top_level_loop_vars, used_key_values),
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 199, in break_up_top_level_values
    sort_config(squished_variants, zip_key_groups)
  File "/home/kmod/miniconda3/lib/python3.8/site-packages/conda_smithy/configure_feedstock.py", line 127, in sort_config
    for pin in list(reversed(sorted(pkg_pins.keys()))):
AttributeError: 'str' object has no attribute 'keys'

It looks like it's trying to interpret part of the config as a dict but it ends up being a single string. Here's the contents of the conda_build_config.yaml:

mpi:
  - mpich    # [not win]
  - openmpi  # [not win]
  - msmpi    # [win]

pin_run_as_build:
  mpich: x.x
  openmpi: x.x
  msmpi: x.x

I checked some other feedstocks and they are written something like this:

pin_run_as_build:
  mpich:
    max_pin: x.x
    min_pin: x.x
  openmpi:
    max_pin: x.x
    min_pin: x.x
  msmpi:
    max_pin: x.x
    min_pin: x.x

which seems to be what conda-smithy is expecting. But @dalcinl and @leofang say that this used to work and is a regression in conda-smithy -- I couldn't find any documentation for conda-smithy indicating whether this is desired behavior or not, so I don't have a personal take on it.

Filed in the feedstock as https://github.com/conda-forge/mpi4py-feedstock/issues/53 but am moving it here at their request.

Installed packages

n/a

Environment info

n/a
isuruf commented 2 years ago

It's not a conda-smithy issue.

pin_run_as_build:
  mpich: x.x
  openmpi: x.x
  msmpi: x.x

is not correct and a change in conda-smithy exposed this.

leofang commented 2 years ago

Thanks @kmod.

@isuruf I disagree, the pin_run_as_build usage is legit, see the documented example and the conda-build source code.