julia v1.9.0 - Githubissues

regro-cf-autotick-bot commented 1 year ago

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

[ ] Dependencies have been updated if changed: see upstream
[ ] Tests have passed
[ ] Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with code>@conda-forge-admin,</codeplease add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
mbedtls	6866

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot. :/ Help is very welcome!

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/4912498197, please use this URL for debugging.}

conda-forge-webservices[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

MilesCranmer commented 1 year ago

Weird, it says the following. Maybe this is a basic fix? @ngam @mkitti

checking file stdlib/LibGit2_jll/src/LibGit2_jll.jl
Hunk #1 FAILED at 21.
1 out of 1 hunk FAILED
checking file stdlib/LibSSH2_jll/src/LibSSH2_jll.jl
checking file stdlib/MbedTLS_jll/src/MbedTLS_jll.jl
checking file stdlib/OpenLibm_jll/src/OpenLibm_jll.jl
checking file stdlib/nghttp2_jll/src/nghttp2_jll.jl
Warning: failed to download source.  If building, will try again after downloading recipe dependencies.
Error was: 
Command '['/opt/conda/bin/patch', '--no-backup-if-mismatch', '--batch', '-Np1', '-i', '/tmp/tmpz8zjimtw/julia-hardcoded-libs.patch.native', '--binary', '--dry-run']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/opt/conda/bin/conda-mambabuild", line 10, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main
    call_conda_build(action, config)
  File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build
    result = api.build(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
    return build_tree(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree
    packages_from_this = build(metadata, stats,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2125, in build
    try_download(m, no_download_source=False, raise_error=True)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/render.py", line 663, in try_download
    raise RuntimeError("Failed to download or patch source. Please see build log for info.")
RuntimeError: Failed to download or patch source. Please see build log for info.

mkitti commented 1 year ago

We would need to rework the patch. I will do it this weekend.

ngam commented 1 year ago

Is the release official yet?
We need to rework quite a few patches and migrate a few core libraries (curl, git, etc.)

mkitti commented 1 year ago

Yes. https://julialang.org/blog/2023/04/julia-1.9-highlights/

ngam commented 1 year ago

@mkitti are you aware of any substantial changes to the build system upstream?

error...

``` 2023-05-12T02:59:12.7693928Z Warning: git information unavailable; versioning information limited 2023-05-12T02:59:13.3453464Z Warning: git information unavailable; versioning information limited 2023-05-12T02:59:16.2894794Z $SRC_DIR/contrib/install.sh 755 $SRC_DIR/usr/bin/julia $PREFIX/bin/ 2023-05-12T02:59:16.2975599Z for suffix in julia ; do \ 2023-05-12T02:59:16.2980975Z for lib in $SRC_DIR/usr/lib/lib${suffix}.*so*; do \ 2023-05-12T02:59:16.2985001Z if [ "${lib##*.}" != "dSYM" ]; then \ 2023-05-12T02:59:16.2989075Z $SRC_DIR/contrib/install.sh 755 $lib $PREFIX/lib ; \ 2023-05-12T02:59:16.2993035Z fi \ 2023-05-12T02:59:16.2996905Z done \ 2023-05-12T02:59:16.3000689Z done 2023-05-12T02:59:16.3127799Z for suffix in libccalltest libllvmcalltest libjulia-internal libjulia-codegen libblastrampoline libdSFMT libuv libLLVM libLLVM-14jl libgfortran libquadmath libstdc++ libgcc_s libgomp libssp libatomic libpthread ; do \ 2023-05-12T02:59:16.3132575Z for lib in $SRC_DIR/usr/lib/${suffix}.*so*; do \ 2023-05-12T02:59:16.3136696Z if [ "${lib##*.}" != "dSYM" ]; then \ 2023-05-12T02:59:16.3140750Z $SRC_DIR/contrib/install.sh 755 $lib $PREFIX/lib/julia ; \ 2023-05-12T02:59:16.3144642Z fi \ 2023-05-12T02:59:16.3155125Z done \ 2023-05-12T02:59:16.3155369Z done 2023-05-12T02:59:16.8241848Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/libpthread.*so*': No such file or directory 2023-05-12T02:59:16.8267343Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libpthread.*so*': No such file or directory 2023-05-12T02:59:16.8275515Z for suffix in libamd libbtf libcamd libccolamd libcholmod libcolamd libklu libldl librbio libspqr libsuitesparseconfig libumfpack libpcre2-8 libgmp libgmpxx libmpfr libssh2 libnghttp2 libmbedtls libmbedcrypto libmbedx509 libcurl libgit2 libz libunwind libopenlibm libopenblas64_ ; do \ 2023-05-12T02:59:16.8276015Z for lib in $SRC_DIR/usr/lib/julia/${suffix}.so*; do \ 2023-05-12T02:59:16.8280415Z if [ "${lib##*.}" != "dSYM" ]; then \ 2023-05-12T02:59:16.8280773Z $SRC_DIR/contrib/install.sh 755 $lib $PREFIX/lib/julia ; \ 2023-05-12T02:59:16.8284844Z fi \ 2023-05-12T02:59:16.8288388Z done \ 2023-05-12T02:59:16.8292166Z done 2023-05-12T02:59:16.8367048Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libbtf.so*': No such file or directory 2023-05-12T02:59:17.6930522Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libbtf.so*': No such file or directory 2023-05-12T02:59:17.6932017Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libklu.so*': No such file or directory 2023-05-12T02:59:17.6933061Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libklu.so*': No such file or directory 2023-05-12T02:59:17.6934419Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libldl.so*': No such file or directory 2023-05-12T02:59:17.6935369Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libldl.so*': No such file or directory 2023-05-12T02:59:17.6936163Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/librbio.so*': No such file or directory 2023-05-12T02:59:17.6937123Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/librbio.so*': No such file or directory 2023-05-12T02:59:17.6946093Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libgmpxx.so*': No such file or directory 2023-05-12T02:59:17.6947111Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libgmpxx.so*': No such file or directory 2023-05-12T02:59:17.6947928Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libz.so*': No such file or directory 2023-05-12T02:59:17.6948881Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libz.so*': No such file or directory 2023-05-12T02:59:17.6949932Z cp: cannot stat '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/usr/lib/julia/libunwind.so*': No such file or directory 2023-05-12T02:59:17.6951037Z chmod: cannot access '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/julia/libunwind.so*': No such file or directory 2023-05-12T02:59:17.6951625Z # Install `7z` into private_libexecdir 2023-05-12T02:59:17.6951908Z $SRC_DIR/contrib/install.sh 755 $SRC_DIR/usr/bin/7z $PREFIX/libexec/julia/ 2023-05-12T02:59:17.6952287Z # Install `lld` into private_libexecdir 2023-05-12T02:59:17.6952589Z $SRC_DIR/contrib/install.sh 755 $SRC_DIR/usr/tools/lld $PREFIX/libexec/julia/ 2023-05-12T02:59:17.6953005Z # Install `dsymutil` into private_libexecdir/ 2023-05-12T02:59:17.6953392Z $SRC_DIR/contrib/install.sh 755 $SRC_DIR/usr/tools/dsymutil $PREFIX/libexec/julia/ 2023-05-12T02:59:17.6953709Z # Copy public headers 2023-05-12T02:59:17.6954065Z cp -R -L $SRC_DIR/usr/include/julia/* $PREFIX/include/julia 2023-05-12T02:59:17.6954400Z # Copy system image 2023-05-12T02:59:17.6954665Z $SRC_DIR/contrib/install.sh 755 $SRC_DIR/usr/lib/julia/sys.so $PREFIX/lib/julia 2023-05-12T02:59:17.6955005Z # Cache stdlibs 2023-05-12T02:59:17.6955302Z [32;1mJULIA[0m [37;1minstall[0m 2023-05-12T02:59:17.6956200Z ERROR: Unable to load dependent library /home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/libjulia-internal.so.1 2023-05-12T02:59:17.6957506Z Message:/home/conda/feedstock_root/build_artifacts/julia_1683858952439/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/libjulia-internal.so.1: cannot open shared object file: No such file or directory 2023-05-12T02:59:17.6960421Z make: *** [Makefile:257: install] Error 1 2023-05-12T02:59:18.7482208Z Traceback (most recent call last): 2023-05-12T02:59:18.7484062Z File "/opt/conda/bin/conda-mambabuild", line 10, in 2023-05-12T02:59:18.7493525Z sys.exit(main()) 2023-05-12T02:59:18.7494130Z File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main 2023-05-12T02:59:18.7528436Z call_conda_build(action, config) 2023-05-12T02:59:18.7530513Z File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build 2023-05-12T02:59:18.7530898Z result = api.build( 2023-05-12T02:59:18.7531692Z File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build 2023-05-12T02:59:18.7551864Z return build_tree( 2023-05-12T02:59:18.7552452Z File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree 2023-05-12T02:59:18.7570556Z packages_from_this = build(metadata, stats, 2023-05-12T02:59:18.7575529Z File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2198, in build 2023-05-12T02:59:18.7575987Z utils.check_call_env(cmd, env=env, rewrite_stdout_env=rewrite_env, 2023-05-12T02:59:18.7576432Z File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 451, in check_call_env 2023-05-12T02:59:18.7592805Z return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs) 2023-05-12T02:59:18.7593543Z File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 427, in _func_defaulting_env_to_os_environ 2023-05-12T02:59:18.7594132Z raise subprocess.CalledProcessError(proc.returncode, _args) 2023-05-12T02:59:18.7594677Z subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/home/conda/feedstock_root/build_artifacts/julia_1683858952439/work/conda_build.sh']' returned non-zero exit status 2. 2023-05-12T03:00:31.2798816Z ##[error]Bash exited with code '1'. 2023-05-12T03:00:31.3132699Z ##[section]Finishing: Run docker build ```

ngam commented 1 year ago

@conda-forge-admin,please rerender

github-actions[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/julia-feedstock/actions/runs/5063111650.

mkitti commented 1 year ago

I have reproduced the issue locally on Linux. Investigating.

mkitti commented 1 year ago

Ok, we actually do produce a working build of Julia 1.9. The problem is that the installation gets screwed up somehow.

(/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env) [conda@be13123b9afa work]$ ./julia 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0
 _/ |\__'_|_|_|\__'_|  |  https://github.com/conda-forge/julia-feedstock
|__/                   |

mkitti commented 1 year ago

It's trying to find /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1 but this file is located in /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/julia/libjulia-internal.so.1.

mkitti commented 1 year ago

The LD_LIBRARY_PATH setting is screwing up julia.

In /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env

(_build_env) [conda@be13123b9afa work]$ export LD_LIBRARY_PATH=${PREFIX}/lib
(_build_env) [conda@be13123b9afa work]$ ./julia
ERROR: Unable to load dependent library /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1
Message:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1: cannot open shared object file: No such file or directory
(_build_env) [conda@be13123b9afa work]$ unset LD_LIBRARY_PATH
(_build_env) [conda@be13123b9afa work]$ ./julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0
 _/ |\__'_|_|_|\__'_|  |  https://github.com/conda-forge/julia-feedstock
|__/                   |

ngam commented 1 year ago

Okay, what we want to do is point julia to the correct path, then?

ngam commented 1 year ago

Playing with LD_LLIBRARY_PATH is likely not a good solution because a lot of things depend on it in the ecosystem

mkitti commented 1 year ago

LD_DEBUG output

``` $ LD_DEBUG=libs LD_LIBRARY_PATH="${PREFIX}/lib" ./julia 18025: find library=libdl.so.2 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/tls/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/tls/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libdl.so.2 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib (RUNPATH from file ./julia) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/tls/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/tls/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/tls/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/tls/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/tls/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/tls/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/x86_64/libdl.so.2 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/libdl.so.2 18025: search cache=/etc/ld.so.cache 18025: trying file=/lib64/libdl.so.2 18025: 18025: find library=libpthread.so.0 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libpthread.so.0 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib (RUNPATH from file ./julia) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libpthread.so.0 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/libpthread.so.0 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/libpthread.so.0 18025: search cache=/etc/ld.so.cache 18025: trying file=/lib64/libpthread.so.0 18025: 18025: find library=libc.so.6 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libc.so.6 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib (RUNPATH from file ./julia) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libc.so.6 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/libc.so.6 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/libc.so.6 18025: search cache=/etc/ld.so.cache 18025: trying file=/lib64/libc.so.6 18025: 18025: find library=libjulia.so.1 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia.so.1 18025: 18025: 18025: calling init: /lib64/libpthread.so.0 18025: 18025: 18025: calling init: /lib64/libc.so.6 18025: 18025: 18025: calling init: /lib64/libdl.so.2 18025: 18025: 18025: calling init: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia.so.1 18025: 18025: find library=libgcc_s.so.1 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libgcc_s.so.1 18025: 18025: 18025: calling init: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libgcc_s.so.1 18025: 18025: find library=libopenlibm.so [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libopenlibm.so 18025: 18025: 18025: calling init: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/julia/libopenlibm.so 18025: 18026: find library=libstdc++.so.6 [0]; searching 18026: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6 18026: 18026: find library=libm.so.6 [0]; searching 18026: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/. (RPATH from file /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6) 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/x86_64/libm.so.6 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/libm.so.6 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./x86_64/libm.so.6 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./libm.so.6 18026: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18026: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libm.so.6 18026: search cache=/etc/ld.so.cache 18026: trying file=/lib64/libm.so.6 18026: 18026: 18026: calling init: /lib64/libm.so.6 18026: 18026: 18026: calling init: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6 18026: 18026: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6: error: symbol lookup error: undefined symbol: GLIBCXX_3.4.31 (fatal) 18025: find library=libstdc++.so.6 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6 18025: 18025: find library=libm.so.6 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./x86_64:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/. (RPATH from file /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/x86_64/libm.so.6 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./tls/libm.so.6 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./x86_64/libm.so.6 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/./libm.so.6 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libm.so.6 18025: search cache=/etc/ld.so.cache 18025: trying file=/lib64/libm.so.6 18025: 18025: 18025: calling init: /lib64/libm.so.6 18025: 18025: 18025: calling init: /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libstdc++.so.6 18025: 18025: find library=libjulia-internal.so.1 [0]; searching 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib (LD_LIBRARY_PATH) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1 18025: search path=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib (RUNPATH from file ./julia) 18025: trying file=/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_build_env/lib/libjulia-internal.so.1 18025: search cache=/etc/ld.so.cache 18025: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path) 18025: trying file=/lib64/tls/x86_64/libjulia-internal.so.1 18025: trying file=/lib64/tls/libjulia-internal.so.1 18025: trying file=/lib64/x86_64/libjulia-internal.so.1 18025: trying file=/lib64/libjulia-internal.so.1 18025: trying file=/usr/lib64/tls/x86_64/libjulia-internal.so.1 18025: trying file=/usr/lib64/tls/libjulia-internal.so.1 18025: trying file=/usr/lib64/x86_64/libjulia-internal.so.1 18025: trying file=/usr/lib64/libjulia-internal.so.1 18025: ERROR: Unable to load dependent library /home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1 Message:/home/conda/feedstock_root/build_artifacts/debug_1685017464512/_h_env/lib/libjulia-internal.so.1: cannot open shared object file: No such file or directory ```

mkitti commented 1 year ago

julia> using Libdl

julia> dllist() .|> println;
/lib64/libdl.so.2
/lib64/libpthread.so.0
/lib64/libc.so.6
/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libjulia.so.1
/lib64/ld-linux-x86-64.so.2
/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libgcc_s.so.1
/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/julia/libopenlibm.so
/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libstdc++.so.6
/lib64/libm.so.6
/home/conda/feedstock_root/build_artifacts/debug_1685017464512/work/usr/bin/../lib/libjulia-internal.so.1

mkitti commented 1 year ago

Playing with LD_LIBRARY_PATH is likely not a good solution because a lot of things depend on it in the ecosystem

Julia seems happy if we do not set LD_LIBRARY_PATH, at least once it's compiled.

Why are we setting this in the first place? https://github.com/conda-forge/julia-feedstock/blob/41f00ab1461dcc29ff22858019094c6c95e0a9bb/recipe/build.sh#L3-L7

mkitti commented 1 year ago

Was that part of some template at some point?

ngam commented 1 year ago

Idk exactly, that precedes both of us. If we can do without them, we should remove them. The conda build process sets most (if not all) the necessary paths

ngam commented 1 year ago

My point is: these paths likely get set anyway. So… we will see. We may need to tinker with julia to make it read from the $PREFIX/lib etc.

ngam commented 1 year ago

@mkitti this may be the issue... https://github.com/JuliaLang/julia/commit/219ef2184723ba0c2bb89af96d79c368c4a0f673

mkitti commented 1 year ago

The next issue I encountered on Linux is libjulia-codegen.so.1:

ERROR: Unable to load dependent library /home/conda/feedstock_root/build_artifacts/julia_1685024684070/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/../lib/julia/libjulia-codegen.so.1
Message:libLLVM-14jl.so: ELF load command address/offset not properly aligned

mkitti commented 1 year ago

This is not the solution, but a hint.

cd $PREFIX
cd lib/julia/
ln -s ../libssl.so.3
ln -s ../libcrypto.so.3
ln -s ../libzstd.so.1
ln -s ../libamd.so.2
ln -s ../libcolamd.so.2 .
ln -s ../libsuitesparseconfig.so.5
ln -s ../libccolamd.so.2
ln -s ../libcamd.so.2
ln -s ../libmetis.so .
ln -s ../libblas.so.3 
ln -s ../liblapack.so.3

The result

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0 (2023-05-07)
 _/ |\__'_|_|_|\__'_|  |  https://github.com/conda-forge/julia-feedstock
|__/                   |

ngam commented 1 year ago

I am okay with a symlink solution as long as we try to figure out how to set it correctly next time...

mkitti commented 1 year ago

Simpler solution:

$ ls -l $PREFIX/lib/julia/{libcholmod.so,libcurl.so}
lrwxrwxrwx 1 conda conda 89 May 27 02:31 /home/conda/feedstock_root/build_artifacts/debug_1685153897226/_h_env/lib/julia/libcholmod.so -> /home/conda/feedstock_root/build_artifacts/debug_1685153897226/_h_env/lib/libcholmod.so.3
lrwxrwxrwx 1 conda conda 86 May 27 02:31 /home/conda/feedstock_root/build_artifacts/debug_1685153897226/_h_env/lib/julia/libcurl.so -> /home/conda/feedstock_root/build_artifacts/debug_1685153897226/_h_env/lib/libcurl.so.4
$ rm $PREFIX/lib/julia/{libcholmod.so,libcurl.so}

mkitti commented 1 year ago

I'm not entirely sure that I am fixing the same issue that CI is encountering.

mkitti commented 1 year ago

I have a local build on Linux. How do I replicate just the test step?

mkitti commented 1 year ago

We are getting closer. There are just some test failures with MPFR now

mkitti commented 1 year ago

Something is different between conda-forge's MPFR build and Julia's MPFR build:

conda-forge MPFR:

julia> import Base.MPFR.libmpfr

julia> let pc = Ref{Ptr{UInt8}}(), fmt = "%.Re", x = big"1.3"
           n = ccall((:mpfr_asprintf,libmpfr), Cint,
               (Ptr{Ptr{UInt8}}, Ptr{UInt8}, Ref{BigFloat}...),
               pc, fmt, x)
           unsafe_string(pc[])
       end
"1e+00"

julia> let pc = Ref{Ptr{UInt8}}(), fmt = "%.78Re", x = big"1.3"
           n = ccall((:mpfr_asprintf,libmpfr), Cint,
               (Ptr{Ptr{UInt8}}, Ptr{UInt8}, Ref{BigFloat}...),
               pc, fmt, x)
           unsafe_string(pc[])
       end
"1.299999999999999999999999999999999999999999999999999999999999999999999999999993e+00"

Julia MPFR:

julia> import Base.MPFR.libmpfr

julia> let pc = Ref{Ptr{UInt8}}(), fmt = "%.Re", x = big"1.3"
           n = ccall((:mpfr_asprintf,libmpfr), Cint,
               (Ptr{Ptr{UInt8}}, Ptr{UInt8}, Ref{BigFloat}...),
               pc, fmt, x)
           unsafe_string(pc[])
       end
"1.299999999999999999999999999999999999999999999999999999999999999999999999999993e+00"

julia> let pc = Ref{Ptr{UInt8}}(), fmt = "%.78Re", x = big"1.3"
           n = ccall((:mpfr_asprintf,libmpfr), Cint,
               (Ptr{Ptr{UInt8}}, Ptr{UInt8}, Ref{BigFloat}...),
               pc, fmt, x)
           unsafe_string(pc[])
       end
"1.299999999999999999999999999999999999999999999999999999999999999999999999999993e+00"

Somehow the default precision is different for mpfr_asprintf.

conda-forge uses

./configure --prefix=$PREFIX \
            --with-gmp=$PREFIX \
            --disable-static

https://github.com/conda-forge/mpfr-feedstock/blob/86b1cebe942849d2f8f40240dbd7a9dc6ab9f8b3/recipe/build.sh#L12-L14

Yggdrasil uses

./configure --prefix=${prefix} --build=${MACHTYPE} --host=${target} --enable-shared --disable-static --with-gmp=${prefix} --enable-thread-safe --enable-shared-cache --disable-float128 --disable-decimal-float

https://github.com/JuliaPackaging/Yggdrasil/blob/a48900da51de224f704843e210913bb39d0c3a90/M/MPFR/build_tarballs.jl#LL23C1-L24C1

mkitti commented 1 year ago

Did the tests fail because we ran out of time?

mkitti commented 1 year ago

Filed https://github.com/conda-forge/mpfr-feedstock/issues/42 with the mpfr-feedstock

$ cat mpfr_test.c
#include <stdio.h>
#include <mpfr.h>

void main() {
    mpfr_t x;
    mpfr_init2(x, 128);
    mpfr_set_str(x, "1.3", 10, MPFR_RNDU);
    mpfr_printf("%.Re\n", x);
}

$ gcc mpfr_test.c -lmpfr -lgmp -o mpfr_test

$ ./mpfr_test 
1.300000000000000000000000000000000000004e+00

$ LD_LIBRARY_PATH=$CONDA_PREFIX/lib ./mpfr_test 
1e+00

mkitti commented 1 year ago

@ngam We could use mpfr 4.1.0. That seems to work fine as far as printing goes. Next build perhaps, if this one works?

mkitti commented 1 year ago

@ngam We have green CI on Julia 1.9

giordano commented 1 year ago

For MPFR you need to apply https://github.com/JuliaLang/julia/pull/49909

mkitti commented 1 year ago

Looks like it's being backported Julia 1.9.1, so I think we'll catch it then.

mkitti commented 1 year ago

@MilesCranmer 1.9.0 should be available shortly in conda-forge

MilesCranmer commented 1 year ago

Nice work!!

MilesCranmer commented 1 year ago

Stress tested the 1.9.0 binary and everything is looking good! (PySR integration tests: https://github.com/MilesCranmer/PySR/actions/runs/5114058241)

conda-forge / julia-feedstock

julia v1.9.0 #245

Pending Dependency Version Updates

Dependency Analysis