search

martinellimarco / t2sz

Compress a file into a seekable zstd with special handling for .tar archives
GNU General Public License v3.0
39 stars 0 forks source link

Allow using stdin #8

Open vasi opened 2 months ago

vasi commented 2 months ago

Currently t2sz mmaps the input file, so it has to be a physical file. This makes t2sz difficult to use in pipelines, where some stream is generated and needs to be compressed, but we don't want the uncompressed stream persisted to disk.

There's no particular reason we couldn't just read blocks into a buffer, though.

martinellimarco commented 2 months ago

Hey there,

You are right. At the moment, this is a design limitation that I should fix by replacing mmap with a buffered input, at least for raw mode. Tar mode requires some additional considerations.

I do plan to address this; however, at this time, I am quite busy with my daily work and can't promise any specific deadline.

It's a feature that I 100% intend to bring to the table, though.

vasi commented 2 months ago

That's reasonable, I know the feeling!