search

Graylog2 / collector-sidecar

Manage log collectors through Graylog
https://www.graylog.org/
Other
268 stars 56 forks source link

Sidecar doesn't load logs list from config #378

Closed kanlas-net closed 5 years ago

kanlas-net commented 5 years ago

Problem description

Sidecar can't start with logs listed in configuration

Steps to reproduce the problem

  1. Use log files list format from example config 
    server_url: "http://graylog.example.local:9000/api/"
server_api_token: "something here"
send_status: true
list_log_files:
- "/var/log/secure"
- "/var/log/messages"
update_interval: 60
log_rotate_max_file_size: "10MiB"
log_rotate_keep_files: 3
  2. Systemd fails to start sidecar daemon. Sidecar debug log: 
    INFO[0000] Using node-id: c2c977fc-cd30-4fbe-8e67-337b046c6624 
INFO[0000] No node name was configured, falling back to hostname 
FATA[0000] Please provide a list of directories for list_log_files.

    Config with line list_log_files: "/var/log/" works well

Environment

kanlas-net commented 5 years ago

My fault, I provide path to files while it needs path to folders

mariussturm commented 5 years ago

Hi @1Stedy, the list_log_files option takes a list of directories not single files. Those directories are then scanned and reported back to the Graylog server. So an admin can see what log files exist and which of them were updated recently. Just provide a list of directories instead and it should work.

on3iropolos commented 4 years ago

@mariussturm - I actually have a use case where I do want to monitor specific log files. I note that when going into the Graylog web portal > Collector Configuration I am able to target specific files to monitor with filebeat, such as below:

filebeat.inputs:

However, if I enter this into the sidecar.yml configuration the service does not start as indicated by @1Stedy . This is problematic as I would prefer to leverage ansible for deploying the targeted log files.

@mariussturm is this something which could be reopened as an issue as I believe it would be very nice to have greater flexibility?

mariussturm commented 4 years ago

@on3iropolos what you are talking about is the actual Filebeat configuration, don't put this into the sidecar.yml. Please read about the basic concept and how the Sidecar is deploying the Filebeat configuration in our docs: http://docs.graylog.org/en/3.1/pages/sidecar.html

on3iropolos commented 4 years ago

@mariussturm Thanks for the explanation. Now I understand that the graylog configuration list_log_files is not which files are logged, but which files to display on the page for the collector id.