xz 5.2.9 breaks conda smithy on windows

[h-vetinari]

I cannot see how from #30, but after the following update:

2022-12-09 09:48:00  (rev 44)
     ca-certificates  {2022.9.24 (conda-forge/win-64) -> 2022.12.7 (conda-forge/win-64)}
     certifi  {2022.9.24 (conda-forge/noarch) -> 2022.12.7 (conda-forge/noarch)}
     conda-forge-pinning  {2022.12.07.18.34.25 (conda-forge/noarch) -> 2022.12.08.16.56.34 (conda-forge/noarch)}
     xz  {5.2.6 (conda-forge/win-64) -> 5.2.9 (conda-forge/win-64)}

trying to rerender locally on windows runs into:

conda smithy rerender
Traceback (most recent call last):
  File "[...]\.conda\envs\builder\Scripts\conda-smithy-script.py", line 5, in <module>
    from conda_smithy.cli import main
  File "[...]\.conda\envs\builder\lib\site-packages\conda_smithy\cli.py", line 13, in <module>
    import conda_build.api
  File "[...]\.conda\envs\builder\lib\site-packages\conda_build\api.py", line 18, in <module>
    from conda_build.config import (Config, get_or_merge_config, get_channel_urls,
  File "[...]\.conda\envs\builder\lib\site-packages\conda_build\config.py", line 19, in <module>
    from .variants import get_default_variant
  File "[...]\.conda\envs\builder\lib\site-packages\conda_build\variants.py", line 19, in <module>
    from conda_build.utils import ensure_list, get_logger, islist, on_win, trim_empty_keys
  File "[...]\.conda\envs\builder\lib\site-packages\conda_build\utils.py", line 11, in <module>
    import libarchive
  File "[...]\.conda\envs\builder\lib\site-packages\libarchive\__init__.py", line 1, in <module>
    from .entry import ArchiveEntry
  File "[...]\.conda\envs\builder\lib\site-packages\libarchive\entry.py", line 6, in <module>
    from . import ffi
  File "[...]\.conda\envs\builder\lib\site-packages\libarchive\ffi.py", line 26, in <module>
    libarchive = ctypes.cdll.LoadLibrary(libarchive_path)
  File "[...]\.conda\envs\builder\lib\ctypes\__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "[...]\.conda\envs\builder\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module '[...]\.conda\envs\builder\Library\bin\archive.dll' (or one of its dependencies). Try using the full path with constructor syntax.

I checked, and envs\builder\Library\bin\archive.dll is there, so it's most likely the "(or one of its dependencies)" part.

Reverting to the last revision of the environment makes conda smithy run again. I then updated {ca-certificates, certifi, conda-forge-pinning} while leaving xz<5.2.9, and it still runs, so it's definitely xz.

@conda-forge/xz @conda-forge/core

[rouault]

It also breaks Windows GDAL builds at linking time, both locally or in CI Cf https://github.com/OSGeo/gdal/actions/runs/3652713343/jobs/6171412987:

2022-12-08T23:20:07.8422656Z    Creating library gdal.lib and object gdal.exp
2022-12-08T23:20:07.8423399Z cpl_compressor.cpp.obj : error LNK2019: unresolved external symbol __imp_lzma_lzma_preset referenced in function "bool __cdecl CPLLZMACompressor(void const *,unsigned __int64,void * *,unsigned __int64 *,char const * const *,void *)" (?CPLLZMACompressor@@YA_NPEBX_KPEAPEAXPEA_KPEBQEBDPEAX@Z)
2022-12-08T23:20:07.8424221Z cpl_compressor.cpp.obj : error LNK2019: unresolved external symbol __imp_lzma_stream_buffer_bound referenced in function "bool __cdecl CPLLZMACompressor(void const *,unsigned __int64,void * *,unsigned __int64 *,char const * const *,void *)" (?CPLLZMACompressor@@YA_NPEBX_KPEAPEAXPEA_KPEBQEBDPEAX@Z)
2022-12-08T23:20:07.8425051Z cpl_compressor.cpp.obj : error LNK2019: unresolved external symbol __imp_lzma_stream_buffer_encode referenced in function "bool __cdecl CPLLZMACompressor(void const *,unsigned __int64,void * *,unsigned __int64 *,char const * const *,void *)" (?CPLLZMACompressor@@YA_NPEBX_KPEAPEAXPEA_KPEBQEBDPEAX@Z)
2022-12-08T23:20:07.8425890Z cpl_compressor.cpp.obj : error LNK2019: unresolved external symbol __imp_lzma_stream_buffer_decode referenced in function "bool __cdecl CPLLZMADecompressor(void const *,unsigned __int64,void * *,unsigned __int64 *,char const * const *,void *)" (?CPLLZMADecompressor@@YA_NPEBX_KPEAPEAXPEA_KPEBQEBDPEAX@Z)
2022-12-08T23:20:07.8426460Z gdal.dll : fatal error LNK1120: 4 unresolved external

Previous build against 5.2.6 was fine.

[isuruf]

Can someone send a PR to mark 5.2.9 broken?

[h-vetinari]

Can someone send a PR to mark 5.2.9 broken?

https://github.com/conda-forge/admin-requests/pull/526

[ocefpaf]

@isuruf any suggestion on how to handle future xz updates? It looks like the build config changed and is causing those breakages.

[JohanMabille]

It looks like both xz and xz-static build the static library on WIndows. I can submit a patch to change this behavior here (and then submit it upstream).