filebeat.yml config file keeps looping on line 18 (i.e password: 'SecretPassword'password: 'SecretPassword'password: 'SecretPassword'..........)

[45triX]

As stated in title Wazuh Manager container keeps on restarting, I think its something to do with a config file but I don't know which one.

My host config is: Debian 12 VM with 12 cores (from a Ryzen 5 5600G) 32gb of DDR4 RAM, 512GB of HDD storage

Here is the logs from the container:

[cont-init.d] 1-config-filebeat: executing... 
Customize Elasticsearch ouput IP
sed: regex input buffer length larger than INT_MAX
[cont-init.d] 1-config-filebeat: exited 4.
[cont-init.d] 2-manager: executing... 
Starting Wazuh v4.7.1...
wazuh-apid: Process 312 not used by Wazuh, removing...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
Started wazuh-apid...
Started wazuh-csyslogd...
Started wazuh-dbd...
2024/01/17 21:33:14 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Started wazuh-integratord...
Started wazuh-agentlessd...
Started wazuh-authd...
Started wazuh-db...
Started wazuh-execd...
Started wazuh-analysisd...
Started wazuh-syscheckd...
Started wazuh-remoted...
Started wazuh-logcollector...
Started wazuh-monitord...
wazuh-modulesd: Process 625 not used by Wazuh, removing...
Started wazuh-modulesd...
Completed.
[cont-init.d] 2-manager: exited 0.
[cont-init.d] done.
[services.d] starting services
starting Filebeat
2024/01/17 21:33:16 wazuh-modulesd:control: INFO: Starting control thread.
2024/01/17 21:33:16 wazuh-modulesd:download: INFO: Module started.
2024/01/17 21:33:16 wazuh-modulesd:database: INFO: Module started.
2024/01/17 21:33:16 wazuh-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2024/01/17 21:33:16 sca: INFO: Loaded policy '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/17 21:33:16 sca: INFO: Starting Security Configuration Assessment scan.
2024/01/17 21:33:16 wazuh-modulesd:syscollector: INFO: Module started.
2024/01/17 21:33:16 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/01/17 21:33:16 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/17 21:33:17 wazuh-modulesd:syscollector: INFO: Evaluation finished.
[services.d] done.
2024/01/17 21:33:22 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/17 21:33:22 sca: INFO: Security Configuration Assessment scan finished. Duration: 6 seconds.
Exiting: error loading config file: yaml: line 18: found character that cannot start any token
Filebeat exited. code=1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 0-wazuh-init: executing... 
/var/ossec/data_tmp/permanent/var/ossec/api/configuration/
The path /var/ossec/api/configuration is already mounted
/var/ossec/data_tmp/permanent/var/ossec/etc/
The path /var/ossec/etc is already mounted
/var/ossec/data_tmp/permanent/var/ossec/logs/
The path /var/ossec/logs is already mounted
/var/ossec/data_tmp/permanent/var/ossec/queue/
The path /var/ossec/queue is already mounted
/var/ossec/data_tmp/permanent/var/ossec/agentless/
The path /var/ossec/agentless is already mounted
/var/ossec/data_tmp/permanent/var/ossec/var/multigroups/
find: '/var/ossec/data_tmp/permanent/var/ossec/var/multigroups/': No such file or directory
The path /var/ossec/var/multigroups is empty, skiped
/var/ossec/data_tmp/permanent/var/ossec/integrations/
The path /var/ossec/integrations is already mounted
/var/ossec/data_tmp/permanent/var/ossec/active-response/bin/
The path /var/ossec/active-response/bin is already mounted
/var/ossec/data_tmp/permanent/var/ossec/wodles/
The path /var/ossec/wodles is already mounted
/var/ossec/data_tmp/permanent/etc/filebeat/
The path /etc/filebeat is already mounted
find: '/proc/227/task/227/fd/6': No such file or directory
find: '/proc/227/task/227/fdinfo/6': No such file or directory
find: '/proc/227/fd/5': No such file or directory
find: '/proc/227/fdinfo/5': No such file or directory
find: '/proc/228/task/228/fd/6': No such file or directory
find: '/proc/228/task/228/fdinfo/6': No such file or directory
find: '/proc/228/fd/5': No such file or directory
find: '/proc/228/fdinfo/5': No such file or directory
Identified Wazuh configuration files to mount...
'/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/etc/ossec.conf'
[cont-init.d] 0-wazuh-init: exited 0.
[cont-init.d] 1-config-filebeat: executing... 
Customize Elasticsearch ouput IP
sed: regex input buffer length larger than INT_MAX
[cont-init.d] 1-config-filebeat: exited 4.
[cont-init.d] 2-manager: executing... 

[45triX]

I found out through a reddit comment that its the the filebeat.yml config file for some reason the password line keeps repeating for some reason i.e:

...
output.elasticsearch:
  hosts: ['https://wazuh.indexer:9200']
  username: 'admin'
  password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'password: 'secretPassword'
  ssl.verification_mode: 'full'
  ssl.certificate_authorities: ['/etc/ssl/root-ca.pem']
  ssl.certificate: '/etc/ssl/filebeat.pem'
  ssl.key: '/etc/ssl/filebeat.key'
  ...

[45triX]

I have no idea what happened but after insanely deleting the password line a million times expecting a different result i actually did get a different result and its working again. I have no idea why the filebeat.yml file was constantly being corrupted but ill post the container logs just in case its helpful:

/var/ossec/data_tmp/permanent/var/ossec/active-response/bin/
The path /var/ossec/active-response/bin is already mounted
/var/ossec/data_tmp/permanent/var/ossec/wodles/
The path /var/ossec/wodles is already mounted
/var/ossec/data_tmp/permanent/etc/filebeat/
The path /etc/filebeat is already mounted
find: '/proc/227/task/227/fd/6': No such file or directory
find: '/proc/227/task/227/fdinfo/6': No such file or directory
find: '/proc/227/fd/5': No such file or directory
find: '/proc/227/fdinfo/5': No such file or directory
find: '/proc/228/task/228/fd/6': No such file or directory
find: '/proc/228/task/228/fdinfo/6': No such file or directory
find: '/proc/228/fd/5': No such file or directory
find: '/proc/228/fdinfo/5': No such file or directory
Identified Wazuh configuration files to mount...
'/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/etc/ossec.conf'
[cont-init.d] 0-wazuh-init: exited 0.
[cont-init.d] 1-config-filebeat: executing... 
Customize Elasticsearch ouput IP
sed: regex input buffer length larger than INT_MAX
[cont-init.d] 1-config-filebeat: exited 4.
[cont-init.d] 2-manager: executing... 
Starting Wazuh v4.7.1...
wazuh-apid: Process 312 not used by Wazuh, removing...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 316, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 313, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 319, removing from /var/ossec/var/run...
Started wazuh-apid...
Started wazuh-csyslogd...
Started wazuh-dbd...
2024/01/21 23:15:33 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Started wazuh-integratord...
Started wazuh-agentlessd...
wazuh-authd: Process 361 not used by Wazuh, removing...
Started wazuh-authd...
Started wazuh-db...
Started wazuh-execd...
Started wazuh-analysisd...
Started wazuh-syscheckd...
Started wazuh-remoted...
Started wazuh-logcollector...
Started wazuh-monitord...
wazuh-modulesd: Process 658 not used by Wazuh, removing...
Started wazuh-modulesd...
Completed.
[cont-init.d] 2-manager: exited 0.
[cont-init.d] done.
[services.d] starting services
starting Filebeat
[services.d] done.
2024/01/21 23:15:35 sca: INFO: Loaded policy '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/21 23:15:35 wazuh-modulesd:database: INFO: Module started.
2024/01/21 23:15:35 wazuh-modulesd:download: INFO: Module started.
2024/01/21 23:15:35 sca: INFO: Starting Security Configuration Assessment scan.
2024/01/21 23:15:35 wazuh-modulesd:control: INFO: Starting control thread.
2024/01/21 23:15:35 wazuh-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2024/01/21 23:15:35 wazuh-modulesd:syscollector: INFO: Module started.
2024/01/21 23:15:35 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/01/21 23:15:35 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/21 23:15:35 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024-01-21T23:15:38.621Z    INFO    instance/beat.go:645    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2024-01-21T23:15:38.652Z    INFO    instance/beat.go:653    Beat ID: 049c9061-8692-4bba-8107-b291d8293a08
2024-01-21T23:15:38.653Z    INFO    [seccomp]   seccomp/seccomp.go:124  Syscall filter successfully installed
2024-01-21T23:15:38.653Z    INFO    [beat]  instance/beat.go:981    Beat info   {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "049c9061-8692-4bba-8107-b291d8293a08"}}}
2024-01-21T23:15:38.654Z    INFO    [beat]  instance/beat.go:990    Build info  {"system_info": {"build": {"commit": "aacf9ecd9c494aa0908f61fbca82c906b16562a8", "libbeat": "7.10.2", "time": "2021-01-12T22:10:33.000Z", "version": "7.10.2"}}}
2024-01-21T23:15:38.654Z    INFO    [beat]  instance/beat.go:993    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":12,"version":"go1.14.12"}}}
2024-01-21T23:15:38.654Z    INFO    [beat]  instance/beat.go:997    Host info   {"system_info": {"host": {"architecture":"x86_64","boot_time":"2024-01-12T01:05:36Z","containerized":false,"name":"wazuh.manager","ip":["127.0.0.1/8","172.18.0.3/16"],"kernel_version":"6.1.0-16-amd64","mac":["02:42:ac:12:00:03"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.6 LTS (Focal Fossa)","major":20,"minor":4,"patch":6,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0}}}
2024-01-21T23:15:38.655Z    INFO    [beat]  instance/beat.go:1026   Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/run/s6/services/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 1276, "ppid": 1270, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2024-01-21T23:15:37.810Z"}}}
2024-01-21T23:15:38.655Z    INFO    instance/beat.go:299    Setup Beat: filebeat; Version: 7.10.2
2024-01-21T23:15:38.794Z    INFO    eslegclient/connection.go:99    elasticsearch url: https://wazuh.indexer:9200
2024-01-21T23:15:38.795Z    INFO    [publisher] pipeline/module.go:113  Beat name: wazuh.manager
2024-01-21T23:15:38.902Z    INFO    beater/filebeat.go:117  Enabled modules/filesets: wazuh (alerts),  ()
2024-01-21T23:15:38.903Z    INFO    instance/beat.go:455    filebeat start running.
2024-01-21T23:15:38.962Z    INFO    memlog/store.go:119 Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=336826
2024-01-21T23:15:39.533Z    INFO    memlog/store.go:124 Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=358653
2024-01-21T23:15:39.534Z    INFO    [registrar] registrar/registrar.go:109  States Loaded from registrar: 1
2024-01-21T23:15:39.534Z    INFO    [crawler]   beater/crawler.go:71    Loading Inputs: 1
2024-01-21T23:15:39.534Z    INFO    log/input.go:157    Configured paths: [/var/ossec/logs/alerts/alerts.json]
2024-01-21T23:15:39.537Z    INFO    [crawler]   beater/crawler.go:141   Starting input (ID: 9132358592892857476)
2024-01-21T23:15:39.537Z    INFO    [crawler]   beater/crawler.go:108   Loading and starting Inputs completed. Enabled inputs: 1
2024-01-21T23:15:39.538Z    INFO    log/harvester.go:302    Harvester started for file: /var/ossec/logs/alerts/alerts.json
2024-01-21T23:15:40.472Z    INFO    [publisher_pipeline_output] pipeline/output.go:143  Connecting to backoff(elasticsearch(https://wazuh.indexer:9200))
2024-01-21T23:15:40.472Z    INFO    [publisher] pipeline/retry.go:219   retryer: send unwait signal to consumer
2024-01-21T23:15:40.476Z    INFO    [publisher] pipeline/retry.go:223     done
2024-01-21T23:15:40.559Z    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2
2024-01-21T23:15:40.560Z    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2
2024-01-21T23:15:40.563Z    INFO    template/load.go:183    Existing template will be overwritten, as overwrite is enabled.
2024-01-21T23:15:40.564Z    INFO    template/load.go:117    Try loading template wazuh to Elasticsearch
2024/01/21 23:15:40 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_ubuntu20-04.yml'
2024/01/21 23:15:40 sca: INFO: Security Configuration Assessment scan finished. Duration: 5 seconds.
2024-01-21T23:15:41.215Z    INFO    template/load.go:109    template with name 'wazuh' loaded.
2024-01-21T23:15:41.215Z    INFO    [index-management]  idxmgmt/std.go:298  Loaded index template.
2024-01-21T23:15:41.218Z    INFO    [publisher_pipeline_output] pipeline/output.go:151  Connection to backoff(elasticsearch(https://wazuh.indexer:9200)) established
2024/01/21 23:15:55 rootcheck: INFO: Ending rootcheck scan.

Edit: This problem is happening everytime the container is restarted, repeatedly fixing the filebeat.yml file as the container starts fixes this

[ezrarieben]

I was having the same issue, but with a custom password set for kibanauser and admin. I ended up switching the INDEXER_PASSWORD to something without special characters, and it has been working stable for me ever since.

[ezrarieben]

I was having the same issue, but with a custom password set for kibanauser and admin. I ended up switching the INDEXER_PASSWORD to something without special characters, and it has been working stable for me ever since.

Judging by the above. This issue may be related to #906.