search

mdsteele / rust-cfb

Rust library for reading/writing Compound File Binary (structured storage) files
MIT License
44 stars 20 forks source link

Compaction after writing shorter streams. #55

Open francisdb opened 3 months ago

francisdb commented 3 months ago

I have the impression that the files never shrink after reducing some streams in size. Have not tested removing streams but could also be affected?

Is there a function available to force a compaction?

Maybe a compact method that looks like the copy code shown in #33?

McSpidey commented 2 months ago

I’m curious about this too. Have you run any tests to confirm it?

mdsteele commented 2 months ago

Correct, the cfb crate currently never shrinks the underlying file. The basic problem here is that cfb::CompoundFile is generic over the underlying reader/writer type (it could be a File, or a Cursor, or a cfb::Stream, or something else entirely), but as far as I know there's no standard Rust trait that provides a generic way to truncate/resize a file-like object. Possibly the best solution is for the cfb crate to provide such a trait, along with implementations for File and other std types.

mdsteele commented 2 months ago

See https://github.com/rust-lang/rfcs/issues/1376

McSpidey commented 2 months ago

Just logging I’ve been doing some basic I/O testing with a simple text file and it’s looking like std::fs::File::set_len (https://doc.rust-lang.org/std/fs/struct.File.html#method.set_len) with seek rewind for the cursor (https://doc.rust-lang.org/std/io/trait.Seek.html#method.rewind) can work to shrink a file. There’s also this truncate in OpenOptons (https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.truncate) but it looks like it does something different.