manojkarthick / pqrs

Command line tool for inspecting Parquet files
Apache License 2.0
294 stars 29 forks source link

[Feature request] "Verify" command #37

Open Hoeze opened 1 year ago

Hoeze commented 1 year ago

Hi, would it be possible to add a "verify" command to check for corrupt parquet files?

SteveLauC commented 1 year ago

This is kinda easy to implement? With a corrupted parquet file, trying to read it would result in:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: General("Invalid Parquet file. Corrupt footer")', src/main.rs:11:55
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/result.rs:1651:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/result.rs:1076:23
   4: rust::main
             at ./src/main.rs:11:23
   5: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.