jj crashes on shallow git repositories (cloned with `--depth 20`)

[ilyagr]

I cloned a git repository with git clone --depth 20 and jj cannot handle it.

For reference, this is the git repository.

Detailed error messages

``` $ RUST_BACKTRACE=1 ../jj/target/debug/jj init --git-repo=. thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NotFound', lib/src/commit.rs:86:49 stack backtrace: 0: rust_begin_unwind at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:584:5 1: core::panicking::panic_fmt at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/panicking.rs:142:14 2: core::result::unwrap_failed at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1785:5 3: core::result::Result::unwrap at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1107:23 4: jujutsu_lib::commit::Commit::parents::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:86:23 5: core::iter::adapters::map::map_fold::{{closure}} at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:84:28 6: core::iter::traits::iterator::Iterator::fold at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:2414:21 7: as core::iter::traits::iterator::Iterator>::fold at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:124:9 8: core::iter::traits::iterator::Iterator::for_each at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:831:9 9: as alloc::vec::spec_extend::SpecExtend>::spec_extend at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_extend.rs:40:17 10: as alloc::vec::spec_from_iter_nested::SpecFromIterNested>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter_nested.rs:62:9 11: as alloc::vec::spec_from_iter::SpecFromIter>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter.rs:33:9 12: as core::iter::traits::collect::FromIterator>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/mod.rs:2649:9 13: core::iter::traits::iterator::Iterator::collect at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:1836:9 14: jujutsu_lib::commit::Commit::parents at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:83:9 15: jujutsu_lib::repo::MutableRepo::add_head::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:704:21 16: as core::ops::function::FnMut>::call_mut at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/boxed.rs:1947:9 17: jujutsu_lib::dag_walk::topo_order_reverse at /usr/local/google/home/ilyagr/dev/jj/lib/src/dag_walk.rs:97:33 18: jujutsu_lib::repo::MutableRepo::add_head at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:700:35 19: jujutsu_lib::git::import_refs at /usr/local/google/home/ilyagr/dev/jj/lib/src/git.rs:99:13 20: jujutsu::cli_util::WorkspaceCommandHelper::import_git_refs_and_head at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:375:9 21: jujutsu::cli_util::WorkspaceCommandHelper::snapshot at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:362:17 22: jujutsu::commands::cmd_init at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:1094:9 23: jujutsu::commands::run_command at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:4529:37 24: jj::run at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:22:5 25: jj::main::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:39 26: core::result::Result::and_then at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1366:22 27: jj::main at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:18 28: core::ops::function::FnOnce::call_once at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/ops/function.rs:248:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. $ RUST_BACKTRACE=1 ../jj/target/debug/jj log thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NotFound', lib/src/commit.rs:86:49 stack backtrace: 0: rust_begin_unwind at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:584:5 1: core::panicking::panic_fmt at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/panicking.rs:142:14 2: core::result::unwrap_failed at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1785:5 3: core::result::Result::unwrap at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1107:23 4: jujutsu_lib::commit::Commit::parents::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:86:23 5: core::iter::adapters::map::map_fold::{{closure}} at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:84:28 6: core::iter::traits::iterator::Iterator::fold at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:2414:21 7: as core::iter::traits::iterator::Iterator>::fold at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:124:9 8: core::iter::traits::iterator::Iterator::for_each at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:831:9 9: as alloc::vec::spec_extend::SpecExtend>::spec_extend at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_extend.rs:40:17 10: as alloc::vec::spec_from_iter_nested::SpecFromIterNested>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter_nested.rs:62:9 11: as alloc::vec::spec_from_iter::SpecFromIter>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter.rs:33:9 12: as core::iter::traits::collect::FromIterator>::from_iter at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/mod.rs:2649:9 13: core::iter::traits::iterator::Iterator::collect at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:1836:9 14: jujutsu_lib::commit::Commit::parents at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:83:9 15: jujutsu_lib::repo::MutableRepo::add_head::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:704:21 16: as core::ops::function::FnMut>::call_mut at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/boxed.rs:1947:9 17: jujutsu_lib::dag_walk::topo_order_reverse at /usr/local/google/home/ilyagr/dev/jj/lib/src/dag_walk.rs:97:33 18: jujutsu_lib::repo::MutableRepo::add_head at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:700:35 19: jujutsu_lib::git::import_refs at /usr/local/google/home/ilyagr/dev/jj/lib/src/git.rs:99:13 20: jujutsu::cli_util::WorkspaceCommandHelper::import_git_refs_and_head at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:375:9 21: jujutsu::cli_util::WorkspaceCommandHelper::snapshot at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:362:17 22: jujutsu::cli_util::CommandHelper::workspace_helper at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:275:9 23: jujutsu::commands::cmd_log at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:2070:29 24: jujutsu::commands::run_command at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:4537:36 25: jj::run at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:22:5 26: jj::main::{{closure}} at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:39 27: core::result::Result::and_then at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1366:22 28: jj::main at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:18 29: core::ops::function::FnOnce::call_once at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/ops/function.rs:248:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. $ rm -r .jj; RUST_BACKTRACE=full ../jj/target/debug/jj init --git-repo=. thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NotFound', lib/src/commit.rs:86:49 stack backtrace: 0: 0x561b46c73f90 - std::backtrace_rs::backtrace::libunwind::trace::h545e191248c18669 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 1: 0x561b46c73f90 - std::backtrace_rs::backtrace::trace_unsynchronized::h71abd128efebd364 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x561b46c73f90 - std::sys_common::backtrace::_print_fmt::ha70541f5338038ba at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:66:5 3: 0x561b46c73f90 - ::fmt::h72be95442683981e at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:45:22 4: 0x561b46c9a3de - core::fmt::write::h9c8fb304d2137801 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/fmt/mod.rs:1202:17 5: 0x561b46c6fec5 - std::io::Write::write_fmt::h144366c4c295a816 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/io/mod.rs:1679:15 6: 0x561b46c75613 - std::sys_common::backtrace::_print::h8148ffddf3fb8bb1 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:48:5 7: 0x561b46c75613 - std::sys_common::backtrace::print::h1100952c4cebfbc4 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:35:9 8: 0x561b46c75613 - std::panicking::default_hook::{{closure}}::h2495bd533729fe5d at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:295:22 9: 0x561b46c752ff - std::panicking::default_hook::h08ed4f1c44c532c3 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:314:9 10: 0x561b46c75cba - std::panicking::rust_panic_with_hook::ha68be045a15b1b9d at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:698:17 11: 0x561b46c75bb7 - std::panicking::begin_panic_handler::{{closure}}::h569bb74226ab900e at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:588:13 12: 0x561b46c7443c - std::sys_common::backtrace::__rust_end_short_backtrace::h158f05b5e681f240 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:138:18 13: 0x561b46c758d2 - rust_begin_unwind at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:584:5 14: 0x561b45e6d2a3 - core::panicking::panic_fmt::hc60cf486a18dc345 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/panicking.rs:142:14 15: 0x561b45e6d3f3 - core::result::unwrap_failed::hb9ba2e3e353aa36f at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1785:5 16: 0x561b46271545 - core::result::Result::unwrap::h5da8c54914505d1d at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1107:23 17: 0x561b46252d1b - jujutsu_lib::commit::Commit::parents::{{closure}}::h1a7839aa07e450e7 at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:86:23 18: 0x561b463e40ac - core::iter::adapters::map::map_fold::{{closure}}::hb99ec2a4a46d9db9 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:84:28 19: 0x561b4625d375 - core::iter::traits::iterator::Iterator::fold::hd3b7abc7016e11e6 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:2414:21 20: 0x561b463d8f15 - as core::iter::traits::iterator::Iterator>::fold::h8f15d39f67103999 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/adapters/map.rs:124:9 21: 0x561b463dddfa - core::iter::traits::iterator::Iterator::for_each::h45a90dc0b5b9dc99 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:831:9 22: 0x561b460eabc2 - as alloc::vec::spec_extend::SpecExtend>::spec_extend::h7a171840e4e401ce at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_extend.rs:40:17 23: 0x561b460c2bd1 - as alloc::vec::spec_from_iter_nested::SpecFromIterNested>::from_iter::h423f15d719e48bc2 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter_nested.rs:62:9 24: 0x561b460ee7b1 - as alloc::vec::spec_from_iter::SpecFromIter>::from_iter::hcfc11a0cd96a5ba0 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/spec_from_iter.rs:33:9 25: 0x561b460e6e58 - as core::iter::traits::collect::FromIterator>::from_iter::hee58cdf04b463822 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/vec/mod.rs:2649:9 26: 0x561b463dc6d1 - core::iter::traits::iterator::Iterator::collect::h1fba8fa6e098cc16 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/iter/traits/iterator.rs:1836:9 27: 0x561b461c19f0 - jujutsu_lib::commit::Commit::parents::h302d7aead1190f12 at /usr/local/google/home/ilyagr/dev/jj/lib/src/commit.rs:83:9 28: 0x561b46284ebc - jujutsu_lib::repo::MutableRepo::add_head::{{closure}}::hfb6e869f5384db52 at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:704:21 29: 0x561b46242de7 - as core::ops::function::FnMut>::call_mut::hb0e1cd40e892c159 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/alloc/src/boxed.rs:1947:9 30: 0x561b460f0369 - jujutsu_lib::dag_walk::topo_order_reverse::h910ac138db510e75 at /usr/local/google/home/ilyagr/dev/jj/lib/src/dag_walk.rs:97:33 31: 0x561b4615d526 - jujutsu_lib::repo::MutableRepo::add_head::hb5f2361f2ddb82f6 at /usr/local/google/home/ilyagr/dev/jj/lib/src/repo.rs:700:35 32: 0x561b460a4221 - jujutsu_lib::git::import_refs::h8cdd771c8181ea50 at /usr/local/google/home/ilyagr/dev/jj/lib/src/git.rs:99:13 33: 0x561b45e7195e - jujutsu::cli_util::WorkspaceCommandHelper::import_git_refs_and_head::hbd06db4b904d12e8 at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:375:9 34: 0x561b45e716a5 - jujutsu::cli_util::WorkspaceCommandHelper::snapshot::h2c56681884d4c39d at /usr/local/google/home/ilyagr/dev/jj/src/cli_util.rs:362:17 35: 0x561b45f1de89 - jujutsu::commands::cmd_init::hc9e622eb6b78d1d3 at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:1094:9 36: 0x561b45f65ad7 - jujutsu::commands::run_command::ha8193b5bdeb85612 at /usr/local/google/home/ilyagr/dev/jj/src/commands.rs:4529:37 37: 0x561b45e6ddcf - jj::run::h4b5697082fe8bbeb at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:22:5 38: 0x561b45e6db9a - jj::main::{{closure}}::h879eb52c29bf4ce8 at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:39 39: 0x561b45e6d76e - core::result::Result::and_then::hdfb60da2da1d2512 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/result.rs:1366:22 40: 0x561b45e6dfa7 - jj::main::h5b8d1cd98ea060a5 at /usr/local/google/home/ilyagr/dev/jj/src/main.rs:27:18 41: 0x561b45e6da3b - core::ops::function::FnOnce::call_once::h30980e4e3d67a24a at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/ops/function.rs:248:5 42: 0x561b45e6d99e - std::sys_common::backtrace::__rust_begin_short_backtrace::h67c4896a8d0916c1 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/sys_common/backtrace.rs:122:18 43: 0x561b45e6d701 - std::rt::lang_start::{{closure}}::ha9e08eb9d5de85f2 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/rt.rs:166:18 44: 0x561b46c6a05f - core::ops::function::impls:: for &F>::call_once::h728b4e90a5e39a28 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/core/src/ops/function.rs:283:13 45: 0x561b46c6a05f - std::panicking::try::do_call::h4feba669c77c7470 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:492:40 46: 0x561b46c6a05f - std::panicking::try::hbc35245f618079fc at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:456:19 47: 0x561b46c6a05f - std::panic::catch_unwind::h99aa000dbffa4e0e at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panic.rs:137:14 48: 0x561b46c6a05f - std::rt::lang_start_internal::{{closure}}::hfa5b3558e81e57d7 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/rt.rs:148:48 49: 0x561b46c6a05f - std::panicking::try::do_call::h1bfa9777096a13a5 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:492:40 50: 0x561b46c6a05f - std::panicking::try::h0212bcd554a851c5 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panicking.rs:456:19 51: 0x561b46c6a05f - std::panic::catch_unwind::h3ec807b79218e75f at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/panic.rs:137:14 52: 0x561b46c6a05f - std::rt::lang_start_internal::hd52e5fa41e14a5d3 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/rt.rs:148:20 53: 0x561b45e6d6da - std::rt::lang_start::h8ca0903aeef263e1 at /rustc/a31705549837622a3c3c27e98c9d8a6d820a984b/library/std/src/rt.rs:165:17 54: 0x561b45e6e081 - main 55: 0x7f567f42920a - __libc_start_call_main at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16 56: 0x7f567f4292bc - __libc_start_main_impl at ./csu/../csu/libc-start.c:389:3 57: 0x561b45e6d5d1 - _start 58: 0x0 - ```

[martinvonz]

We use libgit2, which doesn't support shallow clones. I thought that meant we'd be unlikely to see support for it anytime soon, but it looks like there's actually been some recent progress on it, so that's encouraging :)

[arxanas]

Although libgit2 doesn't support shallow clones, it seems like in this case, the error message is just of a missing parent commit? We'd have to handle that even if libgit2 did support shallow clones, and could handle it right now (somehow; I guess if a parent commit is missing, we should stop iteration and say that the history is shallow?).

[sid-maddy]

We use libgit2, which doesn't support shallow clones. I thought that meant we'd be unlikely to see support for it anytime soon, but it looks like there's actually been some recent progress on it, so that's encouraging :)

@martinvonz, FYI, the PR you mentioned was merged a few months ago.

[martinvonz]

Nice! Now someone will just have to see if it actually just works with jj or we would need to make some changes too.

[Zoybean]

I've reproduced the issue in jj version 0.16.0: Error: Failed to import refs from underlying Git repo

The above-mentioned change to libgit2 was incorporated in libgit2 version 1.7.0. According to the cargo.lock from jj release 0.16.0, it depends on libgit2-sys version 0.16.2+1.7.2 (presumably, this does indeed correctly package libgit2 version 1.7.2, but I didn't bother to check that deep) Seems there will need to be some internal changes to jj in order to accomodate shallow clones.