hvisage
commented
9 years ago I'm looking at (learning GO and) to implement this "option".
To do this, I want to do something like this:
1a) Harvester threads to check for EOF on it's inputs and quit when: i) it's harvesting "-" ii) it's harvesting a file AND the --oneshot flags have been set.
1b) the last harvester thread to close the output channel when it exits.
2) The various channel "listeners" will be checking for close (ie. false status) on their channels (Those are all single instances). Once their channel is closed, they will close their respective output/sending channels and "exit"/quit their thread. This should have a cascading effect starting from the spooler, to the publisher to the registrar. Once the registrar finds a closed channel, it would then quit and the application quit.
Any comments??
driskell
Goodday,
I'd like to have a "one shot" type setting/option.
Reason: I'm importing mass numbers of files from an operation that mv/move/rename the files into the directory I need to process, and then the files stays static, ie. nothing added to them again, but the number grows.
I want to tell logstash-forwarder, to read the files (or even better from stdin), and once the EOF have been reached, to close the file(s), and exit when all of them have been processed. I can then mark/move/etc. the files out of the way, and wait for the next batch and restart/run logstash-forwarder on that set of files. Currently, logstash-forwarder "resets" the stdin input (even though nothing is sending there anymore/closed it down) and then I'll have to kill it myself. This can't be automated reliably to know when logstash-forwarder have sent all the entries reliably to the remote/central logstash instance ;(