vkvam / fpipe

A framework for processing file-like objects with pipes
https://github.com/vkvam/fpipe
MIT License
9 stars 0 forks source link

Split/join and branch/merge #16

Open vkvam opened 4 years ago

vkvam commented 4 years ago

Split/join Split one file into to multiple: By line, count, regex/simple pattern, combinations. A predicate function with a buffer setting would be simpler. Maybe define a few predicate presets. Example: split on new day.

Join could be done in the same way. Example: merge on new month.

Stop previous: Signal previous generators to stop or flush remaining data.

Could be a combination: split count >1 signal stop

Branch/merge Let's say ffmpeg outputs both video and Audio in predictable sized chunks, we could banch audio to one file, video to another, by count. The two files could maybe be merged into ffmpeg again for transcoding.

Split/join: sequential Branch/merge: parallel