search

ostreedev / ostree-rs-ext

Rust library with higher level APIs on top of the core ostree API
Apache License 2.0
72 stars 22 forks source link

lib/tests: Test zstd:chunked format #616

Closed travier closed 2 months ago

travier commented 4 months ago

See: https://github.com/ostreedev/ostree-rs-ext/issues/608#issuecomment-2024089391

travier commented 4 months ago 
failures:

---- test_container_zstd_chunked stdout ----
thread 'test_container_zstd_chunked' panicked at lib/tests/it/main.rs:1341:36:
called `Result::unwrap()` on an `Err` value: Importing

Caused by:
    0: Unencapsulating base
    1: Layer sha256:fd708fa1d754f68b5625c9701c00517146e0e1c9845574c4cc145e18be94cd45
    2: Importing objects
    3: unexpected EOF during skip

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.81/src/error.rs:83:36
   1: ostree_ext::tar::import::Importer::import_objects::{{closure}}::{{closure}}
             at ./src/tar/import.rs:617:32
   2: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:294:13
   3: core::iter::traits::iterator::Iterator::find_map::check::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/iter/traits/iterator.rs:2958:32
   4: core::iter::traits::iterator::Iterator::try_fold
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/iter/traits/iterator.rs:2462:21
   5: core::iter::traits::iterator::Iterator::find_map
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/iter/traits/iterator.rs:2964:9
   6: <core::iter::adapters::filter_map::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::next
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/iter/adapters/filter_map.rs:63:9
   7: ostree_ext::tar::import::Importer::import_objects_impl
             at ./src/tar/import.rs:598:22
   8: ostree_ext::tar::import::Importer::import_objects::{{closure}}
             at ./src/tar/import.rs:619:9
   9: ostree_ext::tar::import::Importer::import_objects
             at ./src/tar/import.rs:609:5
  10: ostree_ext::container::store::ImageImporter::unencapsulate_base::{{closure}}::{{closure}}::{{closure}}
             at ./src/container/store.rs:721:21
  11: ostree_ext::tokio_util::spawn_blocking_cancellable::{{closure}}
             at ./src/tokio_util.rs:51:9
  12: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/blocking/task.rs:42:21
  13: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/core.rs:328:17
  14: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/loom/std/unsafe_cell.rs:16:9
  15: tokio::runtime::task::core::Core<T,S>::poll
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/core.rs:317:13
  16: tokio::runtime::task::harness::poll_future::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:485:19
  17: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panic/unwind_safe.rs:272:9
  18: std::panicking::try::do_call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:554:40
  19: std::panicking::try
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:518:19
  20: std::panic::catch_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panic.rs:142:14
  21: tokio::runtime::task::harness::poll_future
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:473:18
  22: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:208:27
  23: tokio::runtime::task::harness::Harness<T,S>::poll
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:153:15
  24: tokio::runtime::task::raw::RawTask::poll
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/raw.rs:201:18
  25: tokio::runtime::task::UnownedTask<S>::run
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/mod.rs:453:9
  26: tokio::runtime::blocking::pool::Task::run
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/blocking/pool.rs:159:9
  27: tokio::runtime::blocking::pool::Inner::run
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/blocking/pool.rs:513:17
  28: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/blocking/pool.rs:471:13
  29: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys_common/backtrace.rs:155:18
  30: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/thread/mod.rs:529:17
  31: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panic/unwind_safe.rs:272:9
  32: std::panicking::try::do_call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:554:40
  33: std::panicking::try
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:518:19
  34: std::panic::catch_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panic.rs:142:14
  35: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/thread/mod.rs:528:30
  36: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:250:5
  37: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2015:9
  38: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2015:9
  39: std::sys::pal::unix::thread::Thread::new::thread_start
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys/pal/unix/thread.rs:108:17
  40: start_thread
  41: clone3
stack backtrace:
   0: rust_begin_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:647:5
   1: core::panicking::panic_fmt
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/result.rs:1073:23
   4: it::test_container_zstd_chunked::{{closure}}
             at ./tests/it/main.rs:1341:13
   5: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/future/future.rs:124:9
   6: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/future/future.rs:124:9
   7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:659:57
   8: tokio::runtime::coop::with_budget
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/coop.rs:107:5
   9: tokio::runtime::coop::budget
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/coop.rs:73:5
  10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:659:25
  11: tokio::runtime::scheduler::current_thread::Context::enter
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:404:19
  12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:658:36
  13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:737:68
  14: tokio::runtime::context::scoped::Scoped<T>::set
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/context/scoped.rs:40:9
  15: tokio::runtime::context::set_scheduler::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/context.rs:176:26
  16: std::thread::local::LocalKey<T>::try_with
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/thread/local.rs:286:16
  17: std::thread::local::LocalKey<T>::with
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/thread/local.rs:262:9
  18: tokio::runtime::context::set_scheduler
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/context.rs:176:9
  19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:737:27
  20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:646:19
  21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:175:28
  22: tokio::runtime::context::runtime::enter_runtime
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/context/runtime.rs:65:16
  23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/scheduler/current_thread/mod.rs:167:9
  24: tokio::runtime::runtime::Runtime::block_on
             at /var/home/tim/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/runtime.rs:348:47
  25: it::test_container_zstd_chunked
             at ./tests/it/main.rs:1343:5
  26: it::test_container_zstd_chunked::{{closure}}
             at ./tests/it/main.rs:1308:43
  27: core::ops::function::FnOnce::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:250:5
  28: core::ops::function::FnOnce::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

failures:
    test_container_zstd_chunked
cgwalters commented 4 months ago

As for the failure...hmm, looks like it's coming from the tar library. Hmm, like we're running off the end of the stream? AIUI the zstd:chunked is intended to be transparent; the bug could be in either the rust zstd processing or the rust tar library.

cgwalters commented 2 months ago

Narrowed this down to https://github.com/Nullus157/async-compression/issues/271

cgwalters commented 2 months ago

https://github.com/ostreedev/ostree-rs-ext/pull/622