Same inode being used - SinceDB not updated properly with logstash-input-file v4.2.1

[moulisea]

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

• Version: v4.2.1
• Operating System: windows server 2012 R2
• Config File (if you have sensitive info, please remove it):
• Sample Data:
• Steps to Reproduce: when log rotation happens, same inode being used. After reading end of file(xx.log), when log rotation happens, it starts reading(xx.log.2020-10-dd - the file which got rotated). It supposed to read(xx.log file) but thats not happening.

when I delete sincedb file, stop and start logstash with start_position => beginning, the new inode is being generated and the entire records from xx.log is being sent to kafka.

Using logstash 7.9 with logstash-input-file v4.2.1.

This issue is mostly similar to what is mentioned here - https://github.com/logstash-plugins/logstash-input-file/issues/207

Below is the config we use:

input {
  file {
    path => "C:/Users/xx.log"
  }
}

filter {
  mutate {
    remove_field => ["@timestamp", "@version", "path", "host"]
  }
}

output {
  kafka {
    bootstrap_servers => "broker"
    codec => plain { format => "%{message}" }
    topic_id => "topic" 
    client_id => "xxlogs"
    id => "xxlogs"
    jaas_path => ""
    kerberos_config => ""
    sasl_kerberos_service_name => ""    
    sasl_mechanism => ""
    security_protocol => ""
    ssl_endpoint_identification_algorithm => ""
    ssl_key_password => ""
    ssl_keystore_location => ""
    ssl_keystore_password => ""
    ssl_keystore_type => ""
    ssl_truststore_location => ""
    ssl_truststore_password => ""
    ssl_truststore_type => ""
  }
}

[moulisea]

@guyboertje if you have any recommendations, let us know. Thanks

[kares]

this specific case is fixed in 4.2.2 (https://github.com/logstash-plugins/logstash-input-file/pull/276), please give it a go (updated plugin is shipped with LS >= 7.9.3)