awslabs / coldsnap

A command line interface for Amazon EBS snapshots
Apache License 2.0
194 stars 24 forks source link

Downloads will fail if there is a block whoes is greater than 4096 #185

Closed okudajun closed 2 years ago

okudajun commented 2 years ago

The download verb will fail if the target snapshot has a block whose index is greater than 4096. It seems because this line tries to calculate the offset in i32.

        let offset = context.block_index * block_size;

The type of context.block_index and block_size; changed was changed to i32 at https://github.com/awslabs/coldsnap/commit/118e58613b59960ce20aa86c997b951954dbba3d when the AWS SDK coldsnap uses was replaced with AWS's one, and now offset can be up to 2**31-1. The block size is always 512 KiB so far, and block indices of 4096 or larger would cause bigger offsets than 2**31-1.

For example, I have a snapshot that consists of one block whose BlockIndex is 4096.

~/coldsnap$ aws ebs list-snapshot-blocks --snapshot-id snap-0261b433148779711
{
    "Blocks": [
        {
            "BlockIndex": 4096,
            "BlockToken": "ACEBAa+ZbZWpFtWhTvG78qLqkPdvXv+lqBUSfwlbNA0m53UMaQJSMF2nRYTh"
        }
    ],
    "ExpiryTime": 1660512865.211,
    "VolumeSize": 3,
    "BlockSize": 524288
}

If I try to download the snapshot, it will fail.

~/coldsnap$ git status
HEAD detached at v0.4.0
nothing to commit, working tree clean
~/coldsnap$ cargo b
    Finished dev [unoptimized + debuginfo] target(s) in 0.10s
~/coldsnap$ RUST_BACKTRACE=full cargo run -- --region us-west-2 download snap-0261b433148779711 /mnt/test.img
    Finished dev [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/coldsnap --region us-west-2 download snap-0261b433148779711 /mnt/test.img`
  Downloading  [                                                  ] 0/1 (0s)
thread 'main' panicked at 'attempt to multiply with overflow', /home/ubuntu/coldsnap/src/download.rs:344:22
stack backtrace:
   0:     0xaaaad1a7fb20 - std::backtrace_rs::backtrace::libunwind::trace::h2ae93b8a172c7e2c
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0xaaaad1a7fb20 - std::backtrace_rs::backtrace::trace_unsynchronized::h0f06cf40aa7aae7e
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0xaaaad1a7fb20 - std::sys_common::backtrace::_print_fmt::h7f872b389a3ec8cf
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:66:5
   3:     0xaaaad1a7fb20 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h266a3ed794f7087f
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:45:22
   4:     0xaaaad1aa24d8 - core::fmt::write::heec95e0584d70aa9
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/fmt/mod.rs:1197:17
   5:     0xaaaad1a79f34 - std::io::Write::write_fmt::h5f8fc3e116c2d68a
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/io/mod.rs:1672:15
   6:     0xaaaad1a811c4 - std::sys_common::backtrace::_print::hea499b4c6ea71818
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:48:5
   7:     0xaaaad1a811c4 - std::sys_common::backtrace::print::hb12ed1083245fa94
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:35:9
   8:     0xaaaad1a811c4 - std::panicking::default_hook::{{closure}}::ha2e478c9c51e1bc9
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:295:22
   9:     0xaaaad1a80f18 - std::panicking::default_hook::ha8d2f60d84d0d06a
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:314:9
  10:     0xaaaad1a81710 - std::panicking::rust_panic_with_hook::h5d39b9c0d5c93ea7
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:698:17
  11:     0xaaaad1a815bc - std::panicking::begin_panic_handler::{{closure}}::hf31eb739ebb38a74
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:586:13
  12:     0xaaaad1a7ffc8 - std::sys_common::backtrace::__rust_end_short_backtrace::h5b6d7c0cc0584595
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:138:18
  13:     0xaaaad1a81344 - rust_begin_unwind
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:584:5
  14:     0xaaaad0cf0204 - core::panicking::panic_fmt::h91686a7113e4b7c3
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/panicking.rs:142:14
  15:     0xaaaad0cf0120 - core::panicking::panic::h11a54ab253b162fe
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/panicking.rs:48:5
  16:     0xaaaad0da364c - coldsnap::download::SnapshotDownloader::download_block::{{closure}}::h024e282efb813929
                               at /home/ubuntu/coldsnap/src/download.rs:344:22
  17:     0xaaaad0de2198 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h95e880c1d7e45280
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  18:     0xaaaad0da7018 - coldsnap::download::SnapshotDownloader::write_snapshot_blocks::{{closure}}::{{closure}}::{{closure}}::hbef382e5701b76c9
                               at /home/ubuntu/coldsnap/src/download.rs:134:69
  19:     0xaaaad0ddfc40 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h0876dfed01c65528
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  20:     0xaaaad0cfc0b4 - <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next::h58fc83f127c43e28
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17
  21:     0xaaaad0cff18c - futures_util::stream::stream::StreamExt::poll_next_unpin::h6164735306546f34
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9
  22:     0xaaaad0d3f0d0 - <futures_util::stream::stream::for_each_concurrent::ForEachConcurrent<St,Fut,F> as core::future::future::Future>::poll::h78cfc48390a84a42
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/for_each_concurrent.rs:104:19
  23:     0xaaaad0da6804 - coldsnap::download::SnapshotDownloader::write_snapshot_blocks::{{closure}}::ha98a2dfbce6bd222
                               at /home/ubuntu/coldsnap/src/download.rs:145:17
  24:     0xaaaad0de01ac - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h232424f47dcc923e
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  25:     0xaaaad0da5054 - coldsnap::download::SnapshotDownloader::download_to_file::{{closure}}::h3a2892067838da2a
                               at /home/ubuntu/coldsnap/src/download.rs:79:81
  26:     0xaaaad0ddffd0 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1c6845d86849b956
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  27:     0xaaaad0d3a55c - coldsnap::run::{{closure}}::h02404bf241a18bf5
                               at /home/ubuntu/coldsnap/src/bin/coldsnap/main.rs:62:18
  28:     0xaaaad0de39e8 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::heac2709e27ea1f3f
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  29:     0xaaaad0d3b3c8 - coldsnap::main::{{closure}}::hd7bda115dcd96854
                               at /home/ubuntu/coldsnap/src/bin/coldsnap/main.rs:28:26
  30:     0xaaaad0de1b50 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h7b73d35b30c80d16
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/future/mod.rs:91:19
  31:     0xaaaad0d041b0 - tokio::park::thread::CachedParkThread::block_on::{{closure}}::h4f51e26a1eb19619
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/park/thread.rs:263:54
  32:     0xaaaad0d9e1ac - tokio::coop::with_budget::{{closure}}::h716c3c0f45fe02ff
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/coop.rs:102:9
  33:     0xaaaad0d83704 - std::thread::local::LocalKey<T>::try_with::h5fb296ebf702dbf8
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/thread/local.rs:445:16
  34:     0xaaaad0d83210 - std::thread::local::LocalKey<T>::with::h17f6b191b02eb165
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/thread/local.rs:421:9
  35:     0xaaaad0d04050 - tokio::coop::with_budget::hd9c7cb19af694597
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/coop.rs:95:5
  36:     0xaaaad0d04050 - tokio::coop::budget::haf65f52c94e91359
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/coop.rs:72:5
  37:     0xaaaad0d04050 - tokio::park::thread::CachedParkThread::block_on::hc8ed832d4848b124
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/park/thread.rs:263:31
  38:     0xaaaad0d82eb0 - tokio::runtime::enter::Enter::block_on::h8a97993a642fe5ef
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/enter.rs:152:13
  39:     0xaaaad0d91738 - tokio::runtime::thread_pool::ThreadPool::block_on::h19a6f0001ca4cf67
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/thread_pool/mod.rs:90:9
  40:     0xaaaad0dca808 - tokio::runtime::Runtime::block_on::h05205eb41c708c09
                               at /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/mod.rs:484:43
  41:     0xaaaad0d2b9e0 - coldsnap::main::hea54b014f8c09cb2
                               at /home/ubuntu/coldsnap/src/bin/coldsnap/main.rs:28:5
  42:     0xaaaad0d71be8 - core::ops::function::FnOnce::call_once::hbcee081cecd61599
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/ops/function.rs:248:5
  43:     0xaaaad0d20e44 - std::sys_common::backtrace::__rust_begin_short_backtrace::h8e11853a3061cf82
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/sys_common/backtrace.rs:122:18
  44:     0xaaaad0d89a64 - std::rt::lang_start::{{closure}}::hd0460e4dd3def126
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/rt.rs:145:18
  45:     0xaaaad1a74d78 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::ha34b090b924c5319
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/core/src/ops/function.rs:280:13
  46:     0xaaaad1a74d78 - std::panicking::try::do_call::hf0ce84185de29a75
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:492:40
  47:     0xaaaad1a74d78 - std::panicking::try::h6272ec69a2c38c6d
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:456:19
  48:     0xaaaad1a74d78 - std::panic::catch_unwind::hdd65fca3d75df9e2
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panic.rs:137:14
  49:     0xaaaad1a74d78 - std::rt::lang_start_internal::{{closure}}::h2a26dda2a979959f
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/rt.rs:128:48
  50:     0xaaaad1a74d78 - std::panicking::try::do_call::h807c1c750a5bea45
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:492:40
  51:     0xaaaad1a74d78 - std::panicking::try::h9bb8257e9dc87e14
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panicking.rs:456:19
  52:     0xaaaad1a74d78 - std::panic::catch_unwind::h6ab89aba9edfca65
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/panic.rs:137:14
  53:     0xaaaad1a74d78 - std::rt::lang_start_internal::haecda974a76ae3b0
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/rt.rs:128:20
  54:     0xaaaad0d89a30 - std::rt::lang_start::h6f67db8dc9a76ebf
                               at /rustc/6f955bfdfe093edccd6936603d4a8a548293dcf5/library/std/src/rt.rs:144:17
  55:     0xaaaad0d2d5bc - main
  56:     0xffffb9540e10 - __libc_start_main
~/coldsnap$ 
cbgbt commented 2 years ago

Thanks so much for your contribution, @okudajun! This has been released with coldsnap 0.4.1.