Closed dopplershift closed 2 months ago
Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe
) and found it was in an excellent condition.
@conda-forge-admin, please rerender
Hi! This is the friendly automated conda-forge-webservice.
I tried to rerender for you, but it looks like there was nothing to do.
This message was generated by GitHub actions workflow run https://github.com/conda-forge/matplotlib-feedstock/actions/runs/8842524900.
ImportError: /home/conda/feedstock_root/build_artifacts/matplotlib-suite_1714079322516/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.10/site-packages/matplotlib/_path.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3agg10curve3_div4initEdddddd
That looks real 😱
It also looks like atleast some of the builds are trying to build our own freetype.
mplsetup.cfg
is no longer used; config settings need to be passed in to the build backend. https://matplotlib.org/devdocs/api/next_api_changes/development/26621-ES.html
@qulogic Ah, I missed that.
Ping @dopplershift; I don't normally care about conda-forge packages, but some people have asked...
@conda-forge-admin please restart ci
Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe
) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:
{{ stdlib("c") }}
as well. Note that this rule applies to each output of the recipe using a compiler. For further details, please see https://github.com/conda-forge/conda-forge.github.io/issues/2102.__osx
virtual package directly; this should now be done by adding a build dependence on {{ stdlib("c") }}
, and overriding c_stdlib_version
in recipe/conda_build_config.yaml
for the respective platform as necessary. For further details, please see https://github.com/conda-forge/conda-forge.github.io/issues/2102.MACOSX_DEPLOYMENT_TARGET
, to c_stdlib_version
!ImportError: /home/conda/feedstock_root/build_artifacts/matplotlib-suite_1714079322516/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.10/site-packages/matplotlib/_path.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3agg10curve3_div4initEdddddd
That looks real 😱
We reproduced this at the the Summit the other day, and I've opened conda-forge/ctng-compilers-feedstock#135 to get that fixed in the compilers.
Based on discussion in the other thread, it looks like a configuration change is needed
Not entirely sure how the build choose the archiver, but one common way is checking what the environment variable AR
is set to. So maybe something like this?
export AR="${BUILD_PREFIX}/bin/${HOST}-gcc-ar"
IIUC Meson is being used here. If so, it looks like it is possible to configure the ar
used
It looks like GCC_AR
is set to the right static linker, so we could set AR=$GCC_AR
, but I'm hoping for the future Meson can just do that automatically https://github.com/mesonbuild/meson/issues/13324
@conda-forge-admin, please rerender
The windows builds are failing because something is looking for python 3.12 libraries to link to rather than the correct version.
It is also showing:
WARNING: Using legacy MSVC compiler setup. This will be removed in conda-build 4.0. If this recipe does not use a compiler, this message is safe to ignore. Otherwise, use {{compiler('
')}} jinja2 in requirements/build.
but we have both
- {{ compiler('c') }}
- {{ compiler('cxx') }}
already.
[91/101] Linking target src/_c_internal_utils.pypy39-pp73-win_amd64.pyd
FAILED: src/_c_internal_utils.pypy39-pp73-win_amd64.pyd
"link" /MACHINE:x64 /OUT:src/_c_internal_utils.pypy39-pp73-win_amd64.pyd src/_c_internal_utils.pypy39-pp73-win_amd64.pyd.p/_c_internal_utils.cpp.obj "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:src\_c_internal_utils.pypy39-pp73-win_amd64.lib" "ole32.lib" "shell32.lib" "user32.lib" "D:\bld\matplotlib-suite_1719418468749\_h_env\libs\python39.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : fatal error LNK1104: cannot open file 'python312.lib'
Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe
) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:
__osx
virtual package directly; this should now be done by adding a build dependence on {{ stdlib("c") }}
, and overriding c_stdlib_version
in recipe/conda_build_config.yaml
for the respective platform as necessary. For further details, please see https://github.com/conda-forge/conda-forge.github.io/issues/2102.MACOSX_DEPLOYMENT_TARGET
, to c_stdlib_version
!Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe
) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:
c_stdlib_version
below the current global baseline in conda-forge (10.13). If this is your intention, you also need to override MACOSX_DEPLOYMENT_TARGET
(with the same value) locally.Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe
) and found it was in an excellent condition.
@conda-forge-admin, please rerender
well, I have increased the number of failures from when I started trying to fix this.
We are getting this on OSX now:
../meson.build:1:0: ERROR: Could not invoke sanity test executable: [Errno 86] Bad CPU type in executable: '/Users/runner/miniforge3/conda-bld/matplotlib-suite_1719431004502/work/.mesonpy-nxkm6vx1/meson-private/sanitycheckc.exe'.
which just leaves me confused.
Is there something we need to do for Windows builds to pick up freetype
?
Seeing this on CI:
Run-time dependency freetype2 found: NO (tried pkgconfig and cmake)
..\extern\meson.build:11:17: ERROR: Dependency "freetype2" not found, tried pkgconfig and cmake
@conda-forge-admin , please re-render
@conda-forge-admin , please re-render
@conda-forge-admin , please re-render
@conda-forge-admin , please re-render
@conda-forge-admin , please re-render
Hi! This is the friendly automated conda-forge-webservice.
I tried to rerender for you, but it looks like there was nothing to do.
This message was generated by GitHub actions workflow run https://github.com/conda-forge/matplotlib-feedstock/actions/runs/9688515882.
@conda-forge-admin , please re-render
Good news: All *nix builds (including cross-compiled ones) pass 🥳
Bad news: Windows is consistently failing. Having issues picking up system dependencies ( https://github.com/conda-forge/matplotlib-feedstock/pull/373#discussion_r1655870742 ) and before saw issues where builds tried to link the wrong Python library ( https://github.com/conda-forge/matplotlib-feedstock/pull/373#issuecomment-2192415806 ) (saw this with CPython as well) 😞
Not having access to a Windows machine, am ill-positioned to help with the latter. That said, hopefully it is now in a better place for someone to do that debugging 😬
Ok managed to resolve the system dependency issue on Windows
The linking issue remains. Here's an up-to-date example from CI
[30/40] Linking target src/_c_internal_utils.cp39-win_amd64.pyd
FAILED: src/_c_internal_utils.cp39-win_amd64.pyd
"link" /MACHINE:x64 /OUT:src/_c_internal_utils.cp39-win_amd64.pyd src/_c_internal_utils.cp39-win_amd64.pyd.p/_c_internal_utils.cpp.obj "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:src\_c_internal_utils.cp39-win_amd64.lib" "ole32.lib" "shell32.lib" "user32.lib" "D:\bld\matplotlib-suite_1719474041159\_h_env\libs\python39.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : fatal error LNK1104: cannot open file 'python312.lib'
Not sure why it is trying to link Python 3.12 libraries with Python 3.9 (or any other non-3.12 version)
The linker invocation doesn't even mention python312.lib
:
"link" /MACHINE:x64 /OUT:src/_c_internal_utils.cp39-win_amd64.pyd src/_c_internal_utils.cp39-win_amd64.pyd.p/_c_internal_utils.cpp.obj "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:src\_c_internal_utils.cp39-win_amd64.lib" "ole32.lib" "shell32.lib" "user32.lib" "D:\bld\matplotlib-suite_1719474041159\_h_env\libs\python39.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
I think this is coming in from one of the files; I have a vague memory that pybind11 adds the Python library via some kind of #pragma
; do we need to use a Python-specific pybind11
somehow?
I think this is coming in from one of the files; I have a vague memory that pybind11 adds the Python library via some kind of
#pragma
; do we need to use a Python-specificpybind11
somehow?
This might not be relevant, but I have seen similar before on some build systems when meson
, without specific guidance, picks the wrong build
vs host
version of pybind11
. For possible workaround see https://github.com/conda-forge/contourpy-feedstock/blob/9f91c1552136d83d8e6e43c56e4c5f4ede64232f/recipe/build.sh#L12-L18 and https://github.com/emscripten-forge/recipes/blob/main/recipes/recipes_emscripten/contourpy/build.sh for example.
Yeah it is rather odd. Didn't see any other mentions in the logs either
Interesting observation about pybind11. Is this meaningful?
pybind11-config found: NO need ['>=2.6']
...
Run-time dependency pybind11 found: YES 2.13.1
Thought about cross-compilation. However as this is Windows and is being compiled natively, I don't think it is related. Though please let me know if I'm missing something
Or actually, this comes in from the Python headers: https://github.com/python/cpython/blob/1167a9a30b4b2f327ed987e845e378990d1ae6bf/PC/pyconfig.h.in#L310-L336 So we probably have some incorrect include path somewhere.
Maybe pass -Ccompile-args=-v
to build
, so that meson-python tells ninja to compile verbosely, and we can see what include path we have.
The path used for python39.lib
here looks off. Would expect libs
would Library\lib
Snippet from this CI run:
[30/40] "link" /MACHINE:x64 /OUT:src/_c_internal_utils.cp39-win_amd64.pyd src/_c_internal_utils.cp39-win_amd64.pyd.p/_c_internal_utils.cpp.obj "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:src\_c_internal_utils.cp39-win_amd64.lib" "ole32.lib" "shell32.lib" "user32.lib" "D:\bld\matplotlib-suite_1719486440797\_h_env\libs\python39.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
FAILED: src/_c_internal_utils.cp39-win_amd64.pyd
"link" /MACHINE:x64 /OUT:src/_c_internal_utils.cp39-win_amd64.pyd src/_c_internal_utils.cp39-win_amd64.pyd.p/_c_internal_utils.cpp.obj "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:src\_c_internal_utils.cp39-win_amd64.lib" "ole32.lib" "shell32.lib" "user32.lib" "D:\bld\matplotlib-suite_1719486440797\_h_env\libs\python39.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : fatal error LNK1104: cannot open file 'python312.lib'
The corresponding compilation is:
[23/40] "cl.exe"
"-Isrc\_c_internal_utils.pypy39-pp73-win_amd64.pyd.p"
"-Isrc"
"-I..\src"
"-IC:/hostedtoolcache/windows/Python/3.12.4/x64/include"
"-ID:/bld/matplotlib-suite_1719486398479/_h_env/Library/include"
"-ID:\bld\matplotlib-suite_1719486398479\_h_env\Include"
"-DNDEBUG"
"/MD"
"/nologo"
"/showIncludes"
"/utf-8"
"/Zc:__cplusplus"
"/W2"
"/EHsc"
"/std:c++17"
"/permissive-"
"/O2"
"/Gw"
"-DMS_WIN64="
"/d2FH4-"
"-DPY_ARRAY_UNIQUE_SYMBOL=MPL__c_internal_utils_ARRAY_API"
"/Fdsrc\_c_internal_utils.pypy39-pp73-win_amd64.pyd.p\_c_internal_utils.cpp.pdb"
/Fosrc/_c_internal_utils.pypy39-pp73-win_amd64.pyd.p/_c_internal_utils.cpp.obj
"/c"
../src/_c_internal_utils.cpp
That early path of /IC:/hostedtoolcache/windows/Python/3.12.4/x64/include
doesn't look right there.
Can we also check what's in %INCLUDE%
?
From this build am seeing the following
INCLUDE=C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\include;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt;C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um;C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared;C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt;C:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um;D:\bld\matplotlib-suite_1719528423177\_h_env\Library\include
is_x64_arch=true
...
LIB=C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x64;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64;D:\bld\matplotlib-suite_1719528423177\_h_env\Library\lib
LIBPATH=C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x86\store\references;C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22621.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0;C:\Windows\Microsoft.NET\Framework64\v4.0.30319
In all cases except LIBPATH
it is adding the host environment path last
None of them reference hostedtoolcache
. AFAICT it doesn't appear in any environment variable. Just in the flags. So unclear on how it got there
Ok all green 🥳
Please let me know if you need anything else here 🙂
Thanks Tom! 🙏
Looks like CI didn't start when this was merged. So have manually started it
That said, it looks like this may run into the same failure you observed in comment: https://github.com/conda-forge/matplotlib-feedstock/pull/390#issuecomment-2200503373
The macOS and Windows builds passed
Linux builds failed as expected
Since this issue emerged, we have deployed a fix. So restarting failed builds to see if the remaining issues clear up
All builds have now passed and uploaded packages
Note that merging this PR into the rc
branch of this repo pushed 3.9.0 builds onto conda-forge (see https://anaconda.org/conda-forge/matplotlib). Also see https://github.com/conda-forge/matplotlib-feedstock/pull/390#issuecomment-2208282126.
Checklist
0
(if the version changed)conda-smithy
(Use the phrase code>@<space/conda-forge-admin, please rerender in a comment in this PR for automated rerendering)