so3500 / TIL

0 stars 0 forks source link

2024-01-17 #9

Open so3500 opened 5 months ago

so3500 commented 5 months ago

Filebeat

Book

인간력 : 사람을 얻는 힘

함께 자라기 애자일로 가는길

so3500 commented 5 months ago

[ Configure > Inputs > filestream ] https://www.elastic.co/guide/en/beats/filebeat/7.17/filebeat-input-filestream.html

Use the filestream input to read lines from active log files. It is the new, improved alternative to the log input. It comes with various improvments to the existing input:

  1. Checking of close_* options happends out of band. Thus, if an output is blocked, Filebeat can close the reader and avoid keeping too many files open.
  2. Detailed metrics are available for all files that match the paths configuration regardless of the barvester_limit. This way, you can keep track of all files, even ones that are not activly read.
  3. The order of parsers is configurable. So it is possible to parse JSON lines and then aggregate the contents into a multiline event.
  4. Some position updates and metadata changes no longer depend on the publishing pipeline. If the pipeline is blocked some changes are still applied to the registry.
  5. Only the most recent updates are serialized to the registry, In contrast, the log input has to serialize the complete register on each ACK from the outputs. This makes registry updates much quicker with this iput.
  6. The input ensures that only offsets updates are written to the registry append only log. The log writes the complete file state.
  7. Stale entries can be removed from the register, even if there is no active input.
filebeat.inputs:
- type: filestream
  paths:
    - /var/log/messages
    - /var/log/*.log

warning: Each filestream input must have a unique ID. Omitting or changing the filestream ID may cause data duplication. Without a unique ID, filestream is unable to correctly track the state of files.

fields Optional fields that you can specify to add additional information to the output. For example, you might add fields that you can use for filtering log data. Fields can ba scalar values, arrays, dictionaires, or any nested combination of these. By default, the fields that you specify here will be grouped under a fields sub-dic

7.17.16 output > filestream 7.15.2 ouput > log