search

JuliaLang / juliaup

Julia installer and version multiplexer
MIT License
932 stars 81 forks source link

FreeBSD support #872

Open ararslan opened 3 months ago

ararslan commented 3 months ago

This doesn't yet pass tests because the tarball fails to extract and I don't understand why. The URL it's using is correct and I'm able to download and unpack manually.

ararslan commented 3 months ago
Backtrace from tests (click to expand) $ RUST_BACKTRACE=full cargo build warning: file `/usr/home/alex/Projects/juliaup/src/bin/julialauncher.rs` found to be present in multiple build targets: * `bin` target `julia` * `bin` target `julialauncher` Compiling juliaup v1.14.7 (/usr/home/alex/Projects/juliaup) Finished dev [unoptimized + debuginfo] target(s) in 1m 15s $ RUST_BACKTRACE=full cargo test warning: file `/usr/home/alex/Projects/juliaup/src/bin/julialauncher.rs` found to be present in multiple build targets: * `bin` target `julia` * `bin` target `julialauncher` Compiling juliaup v1.14.7 (/usr/home/alex/Projects/juliaup) Finished test [unoptimized + debuginfo] target(s) in 47.42s Running unittests src/lib.rs (target/debug/deps/juliaup-27e8dbb220796096) running 7 tests test operations::tests::match_markers_none_without_markers ... ok test operations::tests::match_markers_returns_correct_indices ... ok test operations::tests::match_markers_returns_err_with_multiple_end ... ok test operations::tests::match_markers_returns_err_with_multiple_start ... ok test operations::tests::match_markers_returns_err_without_end ... ok test operations::tests::match_markers_returns_err_without_start ... ok test utils::tests::test_parse_versionstring ... ok test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s Running unittests src/bin/julialauncher.rs (target/debug/deps/julia-f28678fd7035ba9e) running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Running unittests src/bin/juliaup.rs (target/debug/deps/juliaup-60a8308d342a3057) running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Running tests/channel_selection.rs (target/debug/deps/channel_selection-c6413bcb97490fba) running 1 test test channel_selection has been running for over 60 seconds test channel_selection ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 71.02s Running tests/command_add.rs (target/debug/deps/command_add-1879bb0894255f75) running 1 test test command_add ... FAILED failures: ---- command_add stdout ---- thread 'command_add' panicked at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5: Unexpected failure. code=1 stderr=`````` Installing Julia latest-freebsd-x86_64 Error: Failed to download and extract nightly: Failed to extract downloaded file from url `https://julialangnightlies-s3.julialang.org/bin/freebsd/x86_64/julia-latest-freebsd-x86_64.tar.gz`. Stack backtrace: 0: anyhow::error::::msg at /home/alex/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36 1: juliaup::operations::install_from_url at ./src/operations.rs:463:13 2: juliaup::operations::install_nightly at ./src/operations.rs:534:15 3: juliaup::command_add::add_nightly at ./src/command_add.rs:84:26 4: juliaup::command_add::run_command_add at ./src/command_add.rs:11:16 5: juliaup::main at ./src/bin/juliaup.rs:59:37 6: core::ops::function::FnOnce::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 7: std::sys_common::backtrace::__rust_begin_short_backtrace at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18 8: std::rt::lang_start::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:166:18 9: core::ops::function::impls:: for &F>::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13 10: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 11: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 12: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 13: std::rt::lang_start_internal::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48 14: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 15: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 16: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 17: std::rt::lang_start_internal at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20 18: std::rt::lang_start at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:165:17 19: main 20: __libc_start1 at /usr/src/lib/libc/csu/libc_start1.c:157:7 21: _start at /usr/src/lib/csu/amd64/crt1_s.S:83 ``` ``` command=`JULIAUP_DEPOT_PATH="/tmp/.tmpt52UIs" JULIA_DEPOT_PATH="/tmp/.tmpt52UIs" "/usr/home/alex/Projects/juliaup/target/debug/juliaup" "add" "nightly"` code=1 stdout="" stderr=``` Installing Julia latest-freebsd-x86_64 Error: Failed to download and extract nightly: Failed to extract downloaded file from url `https://julialangnightlies-s3.julialang.org/bin/freebsd/x86_64/julia-latest-freebsd-x86_64.tar.gz`. Stack backtrace: 0: anyhow::error::::msg at /home/alex/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36 1: juliaup::operations::install_from_url at ./src/operations.rs:463:13 2: juliaup::operations::install_nightly at ./src/operations.rs:534:15 3: juliaup::command_add::add_nightly at ./src/command_add.rs:84:26 4: juliaup::command_add::run_command_add at ./src/command_add.rs:11:16 5: juliaup::main at ./src/bin/juliaup.rs:59:37 6: core::ops::function::FnOnce::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 7: std::sys_common::backtrace::__rust_begin_short_backtrace at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18 8: std::rt::lang_start::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:166:18 9: core::ops::function::impls:: for &F>::call_once at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13 10: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 11: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 12: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 13: std::rt::lang_start_internal::{{closure}} at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48 14: std::panicking::try::do_call at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 15: std::panicking::try at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 16: std::panic::catch_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 17: std::rt::lang_start_internal at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20 18: std::rt::lang_start at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:165:17 19: main 20: __libc_start1 at /usr/src/lib/libc/csu/libc_start1.c:157:7 21: _start at /usr/src/lib/csu/amd64/crt1_s.S:83 ``` stack backtrace: 0: 0x32c2adb78076 - std::backtrace_rs::backtrace::libunwind::trace::h7240633c0010cc0c at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5 1: 0x32c2adb78076 - std::backtrace_rs::backtrace::trace_unsynchronized::ha2c7a5c96fbc8505 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x32c2adb78076 - std::sys_common::backtrace::_print_fmt::hd469fe19c1c8394f at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:68:5 3: 0x32c2adb78076 - ::fmt::h579f177009daf63c at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:44:22 4: 0x32c2adba2d30 - core::fmt::rt::Argument::fmt::hd069ebef5ef6ecf2 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/rt.rs:142:9 5: 0x32c2adba2d30 - core::fmt::write::hdac3a427c10c4f97 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/mod.rs:1120:17 6: 0x32c2adb74baf - std::io::Write::write_fmt::hede81785d7f048d1 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/io/mod.rs:1810:15 7: 0x32c2adb77e54 - std::sys_common::backtrace::_print::hfa6dbd0c85e91f76 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:47:5 8: 0x32c2adb77e54 - std::sys_common::backtrace::print::h108ec5c827b50278 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:34:9 9: 0x32c2adb79917 - std::panicking::default_hook::{{closure}}::h963a7edfbc073915 10: 0x32c2adb7961d - std::panicking::default_hook::h8e8085b99e7345eb at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:289:9 11: 0x32c2ad82cb07 - as core::ops::function::Fn>::call::h0bec7cb0f6b7b94a at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2029:9 12: 0x32c2ad82cb07 - test::test_main::{{closure}}::he7ced423a7faeae3 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:138:21 13: 0x32c2adb7a00d - as core::ops::function::Fn>::call::hf66dc46e35826ba0 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2029:9 14: 0x32c2adb7a00d - std::panicking::rust_panic_with_hook::hd05c0541950740f4 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:783:13 15: 0x32c2adb79d52 - std::panicking::begin_panic_handler::{{closure}}::h8bfba3147bcb3614 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:657:13 16: 0x32c2adb78576 - std::sys_common::backtrace::__rust_end_short_backtrace::h163b8cf687c9f743 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:171:18 17: 0x32c2adb79ab0 - rust_begin_unwind at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5 18: 0x32c2adba0d15 - core::panicking::panic_fmt::h8d1bb268d5ef243e at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14 19: 0x32c2ad838f9f - core::panicking::panic_display::h001ba5e4ba9bf4b2 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:196:5 20: 0x32c2ad83c91e - assert_cmd::assert::AssertError::panic::panic_cold_display::h6b37e029948dc0d5 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic.rs:99:13 21: 0x32c2ad83ac27 - assert_cmd::assert::AssertError::panic::ha16d1fcdedc6a331 at /home/alex/.cargo/registry/src/index.crates.io-6f17d22bba15001f/assert_cmd-2.0.13/src/assert.rs:1033:9 22: 0x32c2ad843077 - core::ops::function::FnOnce::call_once::h356ba518d73ccf99 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 23: 0x32c2ad84552f - core::result::Result::unwrap_or_else::h659dcae9e2d4db1c at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1426:23 24: 0x32c2ad83a166 - assert_cmd::assert::Assert::success::h0f4aa7bd61694c1a at /home/alex/.cargo/registry/src/index.crates.io-6f17d22bba15001f/assert_cmd-2.0.13/src/assert.rs:158:9 25: 0x32c2ad7e9e82 - command_add::command_add::hc9a85be4a188313e at /usr/home/alex/Projects/juliaup/tests/command_add.rs:18:5 26: 0x32c2ad7e8547 - command_add::command_add::{{closure}}::h0474660538de43c5 at /usr/home/alex/Projects/juliaup/tests/command_add.rs:5:17 27: 0x32c2ad7ea616 - core::ops::function::FnOnce::call_once::h679daccd76ce5bcf at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 28: 0x32c2ad83238f - core::ops::function::FnOnce::call_once::h73428c76b37975a0 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 29: 0x32c2ad83238f - test::__rust_begin_short_backtrace::he8ead63f1b14979b at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:627:18 30: 0x32c2ad83116e - test::run_test_in_process::{{closure}}::hb19e40931d728242 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:650:60 31: 0x32c2ad83116e - as core::ops::function::FnOnce<()>>::call_once::h137e3c5834ce42f2 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9 32: 0x32c2ad83116e - std::panicking::try::do_call::h981ec909ebe66d3d at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 33: 0x32c2ad83116e - std::panicking::try::h6a8ac355d2c05b51 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 34: 0x32c2ad83116e - std::panic::catch_unwind::he5484ce6066eec19 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 35: 0x32c2ad83116e - test::run_test_in_process::he21ef0196cf852c5 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:650:27 36: 0x32c2ad83116e - test::run_test::{{closure}}::h96161358f8f07f3d at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:573:43 37: 0x32c2ad7f5e16 - test::run_test::{{closure}}::ha89a98e7aac5a3ca at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/test/src/lib.rs:601:41 38: 0x32c2ad7f5e16 - std::sys_common::backtrace::__rust_begin_short_backtrace::hf41028e6b16049a3 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18 39: 0x32c2ad7fb037 - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h7a86dfcd69fe5a2b at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/thread/mod.rs:529:17 40: 0x32c2ad7fb037 - as core::ops::function::FnOnce<()>>::call_once::h7ef355326916622a at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9 41: 0x32c2ad7fb037 - std::panicking::try::do_call::ha2cf9d94c9f940ca at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 42: 0x32c2ad7fb037 - std::panicking::try::hb19633da3c7b0a83 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 43: 0x32c2ad7fb037 - std::panic::catch_unwind::h33d6f55f15e036bb at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 44: 0x32c2ad7fb037 - std::thread::Builder::spawn_unchecked_::{{closure}}::h61726f914b2c59c7 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/thread/mod.rs:528:30 45: 0x32c2ad7fb037 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h79a05a5a909288a3 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5 46: 0x32c2adb7fe85 - as core::ops::function::FnOnce>::call_once::h416b9e2948292dac at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9 47: 0x32c2adb7fe85 - as core::ops::function::FnOnce>::call_once::ha651b059e126a744 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9 48: 0x32c2adb7fe85 - std::sys::unix::thread::Thread::new::thread_start::h83716b035e61a5a7 at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys/unix/thread.rs:108:17 49: 0x32cacfcb0a75 - thread_start at /usr/src/lib/libthr/thread/thr_create.c:290:16 failures: command_add test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 29.38s error: test failed, to rerun pass `--test command_add`
ararslan commented 3 months ago

Actually the unpacking thing might be a red herring possibly caused by with_context attaching the wrong context to a different error. I removed the ? in https://github.com/JuliaLang/juliaup/blob/20a20b97615eedefc5853d61d5d9b30bcc505b44/src/operations.rs#L48 and now the error is that libjulia-codegen.so.1.12 can't load libLLVM-16jl.so. Last I checked (a day or two ago) I didn't see that locally when manually downloading, unpacking, and using nightly. So still doesn't work but... progress?

ararslan commented 3 weeks ago

Swapping the tar crate for binstall-tar, a fork that fixes https://github.com/alexcrichton/tar-rs/issues/295, fixed the primary failure. The test that fails now is related to pulling versiondb from S3, and there isn't one for FreeBSD there. Perhaps I'll separate out the versiondb changes from this PR so that those can be merged first.

ararslan commented 3 weeks ago

...except it didn't fail on CI, huh. The test I mentioned fails locally.

ararslan commented 3 weeks ago

Some to-do items:

ararslan commented 3 weeks ago

binstall-tar has now been synced with upstream tar (https://github.com/cargo-bins/tar-rs/pull/1) and a v0.4.42 release has been published. IMO it seems reasonable to switch to binstall-tar (currently implemented in this PR) until tar has been updated to include the necessary fixes.