julia v1.10.0 - Githubissues

regro-cf-autotick-bot commented 8 months ago

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

[ ] Dependencies have been updated if changed: see upstream
[ ] Tests have passed
[ ] Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with code>@conda-forge-admin,</codeplease add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
mbedtls	6866

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/7337520972, please use this URL for debugging.}

conda-forge-webservices[bot] commented 8 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ngam commented 7 months ago

We need to update the patches

ngam commented 7 months ago

@conda-forge-admin, please rerender

ngam commented 7 months ago

Before merging,

[x] disable threads and bitarray tests (they appear to be flaky in this CI).
[x] Figure out how to handle the issues surrounding suitesparse (we only have v5 in conda-forge, but Julia moved on to v7) --- gotta patch or push suitesparse to 7 in cf

@mkitti would you like to add/subtract more tests here?

mkitti commented 7 months ago

Hmm... looking

github-actions[bot] commented 7 months ago

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

linter: passed
azure: passed

Thus the PR was passing and merged! Have a great day!

github-actions[bot] commented 7 months ago

Hi! This is the friendly conda-forge automerge bot!

Commits were made to this PR after the automerge label was added. For security reasons, I have disabled automerge by removing the automerge label. Please add the automerge label again (or ask a maintainer to do so) if you'd like to enable automerge again!

github-actions[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/julia-feedstock/actions/runs/7765311715.

github-actions[bot] commented 7 months ago

Hi! This is the friendly conda-forge automerge bot!

Commits were made to this PR after the automerge label was added. For security reasons, I have disabled automerge by removing the automerge label. Please add the automerge label again (or ask a maintainer to do so) if you'd like to enable automerge again!

ngam commented 7 months ago

Same blocker as https://github.com/conda-forge/julia-feedstock/issues/260

mkitti commented 7 months ago

Ok, let me see what I can do. Thanks for fixing up the patches.

mkitti commented 7 months ago

My current thought is that we need a script that will set the environment variable LD_DEBUG=all to figure out what is happening.

ngam commented 7 months ago

Btw, I tried building mbedtls on linux but that made it get stuck on some REPL-related compilation. f4ccdf6

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=869979&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97

ngam commented 7 months ago

Actually, let me try something cleaner...

github-actions[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/julia-feedstock/actions/runs/7776873880.

ngam commented 7 months ago

Same issue:

└ Collect (Basic: ◐ , REPL 0/0: ◐ 156) => Execute ◐ 39
fatal: error thrown and no exception handler available.
InitError(mod=:MbedTLS_jll, error=ErrorException("could not load library "libmbedcrypto.so.7"
libmbedcrypto.so.7: cannot open shared object file: No such file or directory"))
ijl_errorf at /usr/local/src/conda/julia-1.10.0/src/rtutils.c:77
ijl_load_dynamic_library at /usr/local/src/conda/julia-1.10.0/src/dlload.c:388
#dlopen#3 at ./libdl.jl:117
dlopen at ./libdl.jl:116 [inlined]
dlopen at ./libdl.jl:116
unknown function (ip: 0x7fbdd270e7dd)
jl_apply at /usr/local/src/conda/julia-1.10.0/src/julia.h:1982 [inlined]
do_call at /usr/local/src/conda/julia-1.10.0/src/interpreter.c:126
eval_value at /usr/local/src/conda/julia-1.10.0/src/interpreter.c:223
eval_stmt_value at /usr/local/src/conda/julia-1.10.0/src/interpreter.c:174 [inlined]
eval_body at /usr/local/src/conda/julia-1.10.0/src/interpreter.c:635
jl_fptr_interpret_call at /usr/local/src/conda/julia-1.10.0/src/interpreter.c:711
__init__ at /home/conda/feedstock_root/build_artifacts/julia_1707082110450/work/usr/share/julia/stdlib/v1.10/MbedTLS_jll/src/MbedTLS_jll.jl:40
jl_apply at /usr/local/src/conda/julia-1.10.0/src/julia.h:1982 [inlined]
jl_module_run_initializer at /usr/local/src/conda/julia-1.10.0/src/toplevel.c:76
_finish_julia_init at /usr/local/src/conda/julia-1.10.0/src/init.c:901
jl_repl_entrypoint at /usr/local/src/conda/julia-1.10.0/src/jlapi.c:725
main at /home/conda/feedstock_root/build_artifacts/julia_1707082110450/work/cli/loader_exe.c:58
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at /home/conda/feedstock_root/build_artifacts/julia_1707082110450/work/usr/bin/julia (unknown line)
└ Collect (Basic: ◐ , REPL 0/0: ◐ 157) => Execute ◐ 39

mkitti commented 7 months ago

That's just the precompilation status. The primary issue is

InitError(mod=:MbedTLS_jll, error=ErrorException("could not load library "libmbedcrypto.so.7"
libmbedcrypto.so.7: cannot open shared object file: No such file or directory"))

mkitti commented 7 months ago

LIBGIT2 and MBEDTLS are coupled. You have to either vendor both of them or neither.

https://github.com/JuliaLang/julia/blob/3d8b10770a7a7317dde13ae36881991db356326c/deps/Makefile#L120-L121

ngam commented 7 months ago

Yeah, I figured we are dealing with the same underlying issue (as the one opened)

github-actions[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/julia-feedstock/actions/runs/7777607904.

mkitti commented 7 months ago

I ran the build locally. Julia runs fine if I run the build in place.

(/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env) [conda@90a0791539ba work]$ ./julia -E "5+5"
10

(/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env) [conda@90a0791539ba work]$ realpath ./julia
/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/julia

The installed version does not work.

(/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env) [conda@90a0791539ba work]$ julia -E "5+5"
fatal: error thrown and no exception handler available.
InitError(mod=:MbedTLS_jll, error=ErrorException("could not load library "libmbedtls.so"
libmbedx509.so.6: cannot open shared object file: No such file or directory"))
ijl_errorf at /usr/local/src/conda/julia-1.10.0/src/rtutils.c:77
ijl_load_dynamic_library at /usr/local/src/conda/julia-1.10.0/src/dlload.c:388
#dlopen#3 at ./libdl.jl:117
dlopen at ./libdl.jl:116 [inlined]
dlopen at ./libdl.jl:116 [inlined]
__init__ at /home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/share/julia/stdlib/v1.10/MbedTLS_jll/src/MbedTLS_jll.jl:42
jfptr___init___89473.1 at /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/julia/sys.so (unknown line)
jl_apply at /usr/local/src/conda/julia-1.10.0/src/julia.h:1982 [inlined]
jl_module_run_initializer at /usr/local/src/conda/julia-1.10.0/src/toplevel.c:76
_finish_julia_init at /usr/local/src/conda/julia-1.10.0/src/init.c:901
jl_repl_entrypoint at /usr/local/src/conda/julia-1.10.0/src/jlapi.c:725
main at /home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/cli/loader_exe.c:58
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at julia (unknown line)

(/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env) [conda@90a0791539ba work]$ which julia
~/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/julia

mkitti commented 7 months ago

From running ./julia out of work, which works.

     10741: find library=libmbedcrypto.so [0]; searching
     10741:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env/lib     (RUNPATH from file ./julia)
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib/libmbedcrypto.so
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedcrypto.so
     10741: 
     10741: 
     10741: calling init: /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedcrypto.so
     10741: 
     10741: find library=libmbedtls.so [0]; searching
     10741:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env/lib     (RUNPATH from file ./julia)
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib/libmbedtls.so
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedtls.so
     10741: 
     10741: find library=libmbedx509.so.6 [0]; searching
     10741:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./tls:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./x86_64:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/.      (RPATH from file /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedtls.so)
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./tls/x86_64/libmbedx509.so.6
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./tls/libmbedx509.so.6
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./x86_64/libmbedx509.so.6
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./libmbedx509.so.6
     10741: 
     10741: 
     10741: calling init: /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/./libmbedx509.so.6
     10741: 
     10741: 
     10741: calling init: /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedtls.so
     10741: 
     10741: find library=libmbedx509.so [0]; searching
     10741:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_build_env/lib     (RUNPATH from file ./julia)
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/work/usr/bin/../lib/libmbedx509.so
     10741:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/lib/libmbedx509.so

From running just julia out of _henv/bin, which fails:

     10735: find library=libmbedcrypto.so [0]; searching
     10735:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/..       (RUNPATH from file julia)
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/libmbedcrypto.so
     10735: 
     10735: 
     10735: calling init: /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/libmbedcrypto.so
     10735: 
     10735: find library=libmbedtls.so [0]; searching
     10735:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/..       (RUNPATH from file julia)
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/libmbedtls.so
     10735: 
     10735: find library=libmbedx509.so.6 [0]; searching
     10735:  search path=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./tls/x86_64:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./tls:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./x86_64:/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/.  (RPATH from file /home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/libmbedtls.so)
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./tls/x86_64/libmbedx509.so.6
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./tls/libmbedx509.so.6
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./x86_64/libmbedx509.so.6
     10735:   trying file=/home/conda/feedstock_root/build_artifacts/debug_1707102000077/_h_env/bin/../lib/julia/./libmbedx509.so.6
     10735:  search cache=/etc/ld.so.cache
     10735:  search path=/lib64/tls:/lib64:/usr/lib64/tls:/usr/lib64        (system search path)
     10735:   trying file=/lib64/tls/libmbedx509.so.6
     10735:   trying file=/lib64/libmbedx509.so.6
     10735:   trying file=/usr/lib64/tls/libmbedx509.so.6
     10735:   trying file=/usr/lib64/libmbedx509.so.6
     10735: 
fatal: error thrown and no exception handler available.
InitError(mod=:MbedTLS_jll, error=ErrorException("could not load library "libmbedtls.so"
libmbedx509.so.6: cannot open shared object file: No such file or directory"))

mkitti commented 7 months ago

If we do the following, then julia will run from $PREFIX.

rm $PREFIX/lib/julia/libmbed*
rm $PREFIX/lib/julia/libssh2.so
rm $PREFIX/lib/julia/libgit2.so

What I do not understand is why are libraries are being moved around by conda like this.

ngam commented 7 months ago

Let's clean this PR up before merging. I can do that later today

ngam commented 7 months ago

@mkitti could you write here your reasoning for the fix? Not important, but I have a feeling it will happen again and it's good to keep a written log

Once we merge this (it's on automerge now) we can attend to other issues

If you want to experiment further with the julialang anaconda channel, we can set up an org and clone stuff from conda-forge (as long as we set the same license and give credit to cf, we should be good)

mkitti commented 7 months ago

Is there an easy way to convert an account into an organization? That was my intention, but I realized I had done it wrong.

My reasoning is entirely empirical. After watching the LD_DEBUG=libs logs I saw the the initial libraries were being detected in the wrong folder, lib/julia. The next set of transitive dependencies were expected to be in the same folder. For the installed dependencies this was not the place to find them. There was an attempt to devendor the libraries. By removing the primary libraries from lib/julia, the lib path would be searched where the primary library AND the transitive dependency can be found.

I'm pretty sure this is ultimately incorrect. The vendored libraries should probably stay vendored so they do not clobber other packages. That said practical issues here outweigh the perfect.

conda-forge / julia-feedstock

julia v1.10.0 #267

Pending Dependency Version Updates

Dependency Analysis