download-snapshot panics before completion

OwenKaneCypher commented 2 months ago

Describe the bug

On latest version (bd14201), when downloading a snapshot it will error when the snapshot is nearly completed.

Steps to reproduce

~~The range doesn't seem to matter so any blocks in the current supported range will error:~~

brontes db download-snapshot -s 20463990 -e 20464000 --brontes-db-path /tmp/brontes

Brontes logs

➜  brontes git:(main) ✗ brontes db download-snapshot -s 20463990 -e 20464000 --brontes-db-path /tmp/brontes

2024-08-26T09:27:14.449136Z  INFO brontes::cli::db::snapshot: new db size 11939mb
2024-08-26T09:27:14.449393Z  INFO brontes::cli::db::snapshot: url="https://data.brontes.xyz/brontes-db-partition-20414677-20465077.tar.gz" size_bytes=11852533760 file_name="brontes-db-partition-20414677-20465077.tar.gz"
2024-08-26T09:27:14.449490Z  INFO brontes::cli::db::snapshot: url="https://data.brontes.xyz/brontes-db-partition-full-range-tables.tar.gz" size_bytes=86761472 file_name="brontes-db-partition-full-range-tables.tar.gz"
2024-08-26T09:27:14.449516Z  INFO brontes::cli::db::snapshot: creating file
2024-08-26T09:27:14.449551Z  INFO brontes::cli::db::snapshot: creating file
download progress
[00:00:00] [--------------------------------------------------------------------------------------------------------] 0 B/82.74 MiB (0.0%) | ETA: 0.0sdownload progress
[00:00:00] [--------------------------------------------------------------------------------------------------------] 0 B/11.04 GiB (0.0%) | ETA: 0.0s2024-08-26T09:43:51.609724Z ERROR brontes_types::executor: Critical task `download_streams` panicked: `called `Result::unwrap()` on an `Err` value: error decoding response body

Caused by:
   0: error reading a body from connection
   1: end of file before message length reached

Location:
    /Dev/brontes/crates/brontes-types/src/buf_writer.rs:112:56`
2024-08-26T09:43:51.611368Z ERROR brontes: ERROR: Critical task `thread` panicked: `panic happened at /Dev/brontes/crates/bin/src/cli/db/snapshot.rs:92:22:
 called `Result::unwrap()` on an `Err` value: error decoding response body

Caused by:
   0: error reading a body from connection
   1: end of file before message length reached

Location:
    /Dev/brontes/crates/brontes-types/src/buf_writer.rs:112:56`

Location:
    /Dev/brontes/crates/bin/src/runner.rs:84:32
Error: program exited via error

Location:
    crates/bin/src/main.rs:43:17

Platform(s)

Mac (Apple Silicon)

What version/commit are you on?

0.1.0-dev

Rust Nightly Version

rustc 1.82.0-nightly (636d7ff91 2024-08-19)

If you've built Brontes from source, provide the full command you used

RUSTFLAGS="-C target-cpu=native" cargo install --path crates/bin --profile maxperf

Code of Conduct

[X] I agree to follow the Code of Conduct

OwenKaneCypher commented 2 months ago

Update: I changed the range after looking at others able to successfully download parts of the snapshot. I was able to download the range 20363990-20364000. However, 20463990-20464000 still errors

Will-Smith11 commented 2 months ago

Update: I changed the range after looking at others able to successfully download parts of the snapshot. I was able to download the range 20363990-20364000. However, 20463990-20464000 still errors

hmm interesting. guessing data is partially corrupted, going to be uploading the full range next couple of days so hoping that will fix this but will leave open now as a reminder

SorellaLabs / brontes