eBay / tsv-utils

eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.
https://ebay.github.io/tsv-utils/
Boost Software License 1.0
1.42k stars 80 forks source link

Line buffering updates and tsv-select support #334

Closed jondegenhardt closed 3 years ago

jondegenhardt commented 3 years ago

This PR makes some updates to the line-buffered support added in PR #333. That PR added some general support in BufferedByLine and BufferedOutoutRange and added a --line-buffered option to tsv-filter.

This PR extends this by adding support to ByLineSourceRange. This was needed to add support to tsv-select.

Some changes were made to the flushing logic in BufferedOutputRange. This causes a flush to the output stream (eg. stdout) more often, but improves operation in unix pipelines (both header line flushing and line-buffered support). It also improves error messages. It was a small performance hit to csv2tsv, but the overall effect is worthwhile.