StdinStreamRunner considers it reaches EOF when the stdin is not ready.
This results stream2es finishes before the previous program finishes the stream, especially when the program is slow.
This change blocks the runner while reading stdin, and considers reaching EOF only after stdin is closed.
Steps to reproduce problems:
Create a script testoutput.sh:
#!/bin/sh
for i in $(seq 10); do
TIMESTAMP=`date +%Y-%m-%dT%H:%M:%SZ`
printf '{"@timestamp": "%s", "duration": 10}\n' "${TIMESTAMP}"
sleep 1 # demonstrating a really slow program.
done
run stream2es:
$ ./testoutput.sh |./stream2es stdin
Result (actual, unexpected): stream2es finishes before processing all the outputs from the script like this (only 3 lines are processed):
2015-12-08T23:10:54.503+0900 INFO 00:00.172 17.4d/s 1.5K/s (0.0mb) indexed 3 streamed 3 errors 0
2015-12-08T23:10:54.503+0900 INFO done
Expected result (result after this change): all 10 lines are processed:
2015-12-08T23:12:11.330+0900 INFO 00:08.099 1.2d/s 0.1K/s (0.0mb) indexed 10 streamed 10 errors 0
2015-12-08T23:12:11.345+0900 INFO done
StdinStreamRunner
considers it reaches EOF when the stdin is not ready. This resultsstream2es
finishes before the previous program finishes the stream, especially when the program is slow.This change blocks the runner while reading stdin, and considers reaching EOF only after stdin is closed.
Steps to reproduce problems:
Create a script
testoutput.sh
:stream2es
:$ ./testoutput.sh |./stream2es stdin
Result (actual, unexpected): stream2es finishes before processing all the outputs from the script like this (only 3 lines are processed):
Expected result (result after this change): all 10 lines are processed: