This PR adds a --line-buffered option to tsv-filter. When run with this, tsv-filter will read and write each line without buffering. This is useful when processing input streams that receive data slowly. However, it is a performance hit when data is available in bulk.
This was implemented by added support to BufferedInputRange and BufferedOutputRange, then invoking these facilities with the proper parameters from tsv-filter proper. This also enabled getting rid of the ad-hoc scheme in tsv-filter for ensuring lines were occasionally written when run on slow input streams.
Line buffered support will be added to other tools in the future. One step that needs to occur for some of the tools is to add support to ByLineSourceRange.
This PR adds a
--line-bufferedoption totsv-filter. When run with this,tsv-filterwill read and write each line without buffering. This is useful when processing input streams that receive data slowly. However, it is a performance hit when data is available in bulk.This was implemented by added support to
BufferedInputRangeandBufferedOutputRange, then invoking these facilities with the proper parameters fromtsv-filterproper. This also enabled getting rid of the ad-hoc scheme intsv-filterfor ensuring lines were occasionally written when run on slow input streams.Line buffered support will be added to other tools in the future. One step that needs to occur for some of the tools is to add support to
ByLineSourceRange.