Reading larger files fails at branch level on collect

[denehoffman]

I've been seeing this error recently when trying to read from larger files. I haven't tried a bunch of different sizes, but the largest I've successfully read was 63Mb, while a file with 832Mb fails with the following code:

use oxyroot::RootFile;
fn main() {
    let tree = RootFile::open("large_file.root").unwrap().get_tree("kin").unwrap();
    // read large_file.root's "kin" tree
    let e_beam_iter = tree.branch("E_Beam").unwrap().as_iter::<f32>().unwrap();
    // this is fine
    let data: Vec<f32> = e_beam_iter.collect(); // this fails
}

The backtrace reads:

thread 'main' panicked at /home/dene/.cargo/registry/src/index.crates.io-6f17d22bba15001f/oxyroot-0.1.24/src/rtree/basket.rs:103:46:
called `Result::unwrap()` on an `Err` value: RCompress(Io(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" }))
stack backtrace:
   0:     0x5bc434977275 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hbaf8fc189101dfe8
   1:     0x5bc43499a0fb - core::fmt::write::hf660ac4eb6f2add1
   2:     0x5bc43497511f - std::io::Write::write_fmt::h0aa23be0bd9cfa0c
   3:     0x5bc4349783c1 - std::panicking::default_hook::{{closure}}::h9a3ea3db17847643
   4:     0x5bc43497809c - std::panicking::default_hook::h78fb7d1fff374982
   5:     0x5bc4349789b1 - std::panicking::rust_panic_with_hook::h29b0bcc55ace85de
   6:     0x5bc4349788a7 - std::panicking::begin_panic_handler::{{closure}}::he940321d814d7e45
   7:     0x5bc434977739 - std::sys::backtrace::__rust_end_short_backtrace::h8333ddb76a6f1f6a
   8:     0x5bc434978534 - rust_begin_unwind
   9:     0x5bc434997b93 - core::panicking::panic_fmt::h3883036404a69a62
  10:     0x5bc434998006 - core::result::unwrap_failed::h6efaafb729d50e24
  11:     0x5bc43485ee08 - oxyroot::rtree::basket::Basket::raw_data::hbb2439a0c1eeeac9
  12:     0x5bc434838984 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::next::h5f4176d8a316f6e7
  13:     0x5bc43483207e - <core::iter::adapters::flatten::FlatMap<I,U,F> as core::iter::traits::iterator::Iterator>::next::hb29c66c363839753
  14:     0x5bc434835981 - <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hceb0068ad6fdfcdc
  15:     0x5bc4348370cc - resonaria::main::he92cd2e2722fd30e
  16:     0x5bc4348373f3 - std::sys::backtrace::__rust_begin_short_backtrace::ha3d6ad1e23c76c67
  17:     0x5bc4348373e9 - std::rt::lang_start::{{closure}}::h1028faad70c2cac5
  18:     0x5bc434972610 - std::rt::lang_start_internal::h8a402ec7b92c5251
  19:     0x5bc4348373d5 - main
  20:     0x79a24b6d4c88 - <unknown>
  21:     0x79a24b6d4d4c - __libc_start_main
  22:     0x5bc434831105 - _start
  23:                0x0 - <unknown>

This file isn't corrupt, it loads fine via python's uproot, but I'm not sure what's going wrong here. I can send the data file if there is some way you'd prefer to receive an 800Mb file.

[m-dupont]

I couldn't reproduce the issue using a 1.2GB file (a single tree with one branch of f32). Could you share a ROOT script that generates a large problematic file?

[denehoffman]

I'll work on that, yes!