Open mcint opened 1 year ago
I've resolved my use, with the line-buffered output and while read loop for line buffered input, but I leave this here so the incremental saving or line-buffered use-case can be explicitly handled or rejected (or deferred).
I'm trying to use sqlite-utils to data generated incrementally. There are a few aspects of this that I don't currently know how to handle. I would like an option to apply writes incrementally, line-by-line as they are received. I would like an option to echo incremental progress. And, it would be nice to have
In particular, I'm using CoreLocationCLI -w -j to generate, newline-delimited JSON.
One variant of the command
stdbuf -oL CoreLocationCLI -w -j | pee 'sqlite-utils insert loc.db loc -' nl
pee
, frommoreutils
, is liketee
but spawns and pipes to the processes created by invoking each of its arguments, so, for gratuitous demonstration,pee 'sponge out.log' cat
would behave liketee
.It looks like I can get what I want with:
stdbuf -oL CoreLocationCLI -w -j | while read line; do <<<"$line" sqlite-utils insert loc.db loc -; echo "$line"; done | nl