search

conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
149 stars 176 forks source link

Non-deterministic behavior when rerendering pytensor-suite-feedstock #1782

Open maresb opened 12 months ago

maresb commented 12 months ago

Solution to issue cannot be found in the documentation.

Issue

See https://github.com/conda-forge/pytensor-suite-feedstock/issues/54 for technical details of my debugging attempts.

In short, since several months, running conda-smithy rerender randomly skips .ci-support files which shouldn't be skipped. It took me several hours with the debugger to get an idea about what's going on.

To reproduce:

  1. Clone conda-forge/pytensor-suite-feedstock
  2. Run conda-smithy rerender
  3. Run git status and check for deleted files under .ci_support/
  4. Repeat, and observe files appearing and disappearing from .ci_support/ at random.

My rough understanding is that conda-smithy calls conda-build in order to determine whether to render a group like osx-arm64. Associated to this group is a set of Python versions ranging from 3.8 to 3.11. The recipe contains skip: true # [py<39]. In evaluating whether or not to skip osx-arm64, a Python version is selected at random. Whenever 3.8 comes up, it skips osx-arm64 and causes the corresponding .ci-support files to go missing.

Installed packages

conda-build 3.25.0, conda-smithy 3.26.2, and conda-forge-pinning 2023.10.07.18.27.00

Environment info

N/A
mbargull commented 8 months ago

Hi @maresb, the conda-smithy part of this issue is likely fixed via gh-1815 in a future conda-smithy release. Another part of the issue which makes it much more likely to occur for pytensor-suite-feedstock is due to its usage of the py selector in the top-level skip line of the recipe in combination with python being only used for the outputs. I've suggested a change to better handle this case in conda-build via https://github.com/conda/conda-build/pull/5139 . See that pull request's description for more details on why it happens and also how you could work around it for now in the recipe.

maresb commented 8 months ago

Thanks @mbargull for helping me solve #54. That was the functional problem I was trying to solve.

Please feel free to close this issue whenever you see fit. Thanks so much for tracking down these really subtle problems!!!