"has no content yet, skipping" gets logged when File is older than ignore_older

[marcohald]

The Log should not get spammed with "has no content yet, skipping" if the file is older than ignore_older. I added ignore_inactive: since_last_start but that also changed nothing

Please include configurations and logs if available. Config Snippet:

  inputs:
  - fields_under_root: true
    id: application
    ignore_inactive: since_last_start
    ignore_older: 168h
    parsers:
    - multiline:
        match: after
        negate: true
        pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}
        type: pattern
    paths:
    - /etc/application/*/upload/logs/*.log
    take_over: true
    type: filestream

Log:

Oct 17 13:17:23 app.example.com filebeat[110189]: 
{"log.level":"warn","@timestamp":"2023-10-17T13:17:23.450+0200","log.logger":"file_watcher","log.origin":{"file.name":"filestream/fswatch.go","file.line":204},"message":"file \"/etc/application/prod/upload/logs/check_installation_web_2019-08-21_093048.log\" has no content yet, skipping","service.name":"filebeat","ecs.version":"1.6.0"}
:~# ls -lah /etc/application/prod/upload/logs/check_installation_web_2019-08-21_093048.log
-rw-r--r-- 1 www-data www-data 0 Aug 21  2019 /etc/application/prod/upload/logs/check_installation_web_2019-08-21_093048.log

For confirmed bugs, please report:

• Version: filebeat version 8.10.3 (amd64), libbeat 8.10.3 [37113021c2d283b4f5a226d81bc77d9af0c8799f built 2023-10-05 05:52:27 +0000 UTC]
• Operating System: Debian GNU/Linux 11 (bullseye)
• Steps to Reproduce:
  1. Create an File that is older than ignore_older
  2. Run the Config with at least log level warning

[botelastic[bot]]

This issue doesn't have a Team:<team> label.

[Bobo357]

IMHO that "file xxx has no content yet, skipping" message does not belong to warning level, should be info. Regardless of ignore_older setting. This is very annoying, because filebeat spams VERY OFTEN about Ubuntu system log files like kern.log and dpkg.log that receive messages quite rarely and therefore are empty most of the time after logrotate has rotated them.

Please fix this.

[draganHR]

Trying to drop the message with something like:

processors:
 - drop_event:
     when:
       equals:
         service.name: "filebeat"

...but it doesn't work. Are processors even applied to filebeat logs?

[Alphayeeeet]

Could this log level changed to info or even debug to prevent log spamming?

[marcohald]

The Change of the Logging Level https://github.com/elastic/beats/pull/38421 does not solve this issue . But it is easier to handle now, as it is not logged in any log level we use in production.