search

driskell / log-courier

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.
Other
419 stars 107 forks source link

log-courier doesn't end if started with -stdin #353

Closed blablup closed 7 years ago

blablup commented 7 years ago

Hello,

In my case log-courier never ends (unless I hit STRG-C) if it is started with command line -stdin

My Config:

general:
  persist directory: /var/lib/log-courier
  log level: "debug"
network:
  servers:
    - localhost:7777
  transport: tcp
stdin:
  fields:
    type: test

my command to test

echo "test";echo "test" |log-courier -config ./test.yaml -stdin

the output

2017/01/11 15:24:39.061550 Log Courier version 2.0.4 pipeline starting
2017/01/11 15:24:39.061739 [Random] Randomly selected new endpoint: localhost:7777
2017/01/11 15:24:39.061909 Pipeline ready
2017/01/11 15:24:39.062604 [localhost:7777] Attempting to connect to 127.0.0.1:7777 (localhost)
2017/01/11 15:24:39.062972 Started harvester: stdin
2017/01/11 15:24:39.063430 [localhost:7777] Connected to 127.0.0.1:7777 (localhost)
2017/01/11 15:24:39.063513 [localhost:7777] Endpoint is ready
2017/01/11 15:24:39.063528 [localhost:7777] Starting keepalive timeout
2017/01/11 15:24:39.063545 Timeout timer reset - due at 2017-01-11 15:39:39.063532737 +0100 CET
2017/01/11 15:24:40.063758 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:41.063912 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:42.064062 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:43.064221 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:44.063503 Spooler flushing 2 events due to spool timeout exceeded
2017/01/11 15:24:44.063603 [localhost:7777] Sending payload 56847afe9799739b3d9677972a3b58ef (2 events)
2017/01/11 15:24:44.063866 Timeout timer reset - due at 2017-01-11 15:24:59.063846548 +0100 CET
2017/01/11 15:24:44.064379 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:44.065721 [localhost:7777] Acknowledgement received for payload 56847afe9799739b3d9677972a3b58ef sequence 2
2017/01/11 15:24:44.065751 [localhost:7777] Average latency per event: 1.07 ms
2017/01/11 15:24:44.065766 Timeout timer reset - due at 2017-01-11 15:39:44.065756728 +0100 CET
2017/01/11 15:24:44.065816 Registrar received offsets for 2 log entries
2017/01/11 15:24:45.064512 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:46.064766 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:47.064989 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:48.065203 Stopping harvest of %!s(MISSING); EOF reached
^C2017/01/11 15:24:48.353433 Initiating shutdown

here I did hit STRG-C 

2017/01/11 15:24:48.353460 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:48.353483 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:48.353492 Stopping harvest of %!s(MISSING); EOF reached
2017/01/11 15:24:48.353499 Stopping harvest of %!s(MISSING); EOF reached

This repeats 11770 times

2017/01/11 15:24:49.065199 Harvester for stdin exiting
2017/01/11 15:24:49.065334 Aborted reading from stdin at offset 10
2017/01/11 15:24:49.065499 [localhost:7777] Endpoint is now shutting down
2017/01/11 15:24:49.065586 Spooler exiting
2017/01/11 15:24:49.065658 Stdin registrar exiting
2017/01/11 15:24:49.068210 [localhost:7777] Disconnected from 127.0.0.1:7777 (localhost)
2017/01/11 15:24:49.069582 [localhost:7777] Endpoint has finished
2017/01/11 15:24:49.069613 Publisher exiting
2017/01/11 15:24:49.069636 Exiting

Why is log-courier not ending properly?

Jesai

driskell commented 7 years ago

Will push the fix to repositories in the coming days and close when it's live. If you're building from source, give master branch a go :)

driskell commented 7 years ago

Released 2.0.5 to repositories yesterday!