S3 dynamic folder naming in yyyy/mm/dd

[jaeniale-fd]

I am using Logstash with the s3 input plugin to copy and rename incoming files. I want to use the time when logstash picks up the file as a prefix of the new filename.

• Version: 7.5.1
• Operating System: Mac / Linux Docker
• Config File:

  input {
  s3 {
  access_key_id => "some_key"
  secret_access_key => "some_access_key"
  region => "some_region"
  bucket => "mybucket"
  interval => "10"
  sincedb_path => "/tmp/sincedb_something"
  backup_add_prefix =>'%{+yyyy.MM.dd.HH}'
  backup_to_bucket => "mybucket"
  additional_settings => {
    force_path_style => true
    follow_redirects => false
              }
  }
  }

  the current result is:

test.txt is coming in, logstash picks it up and puts it in the same folder as %{+yyyy.MM.dd.HH}test.txt

Is there a way to make backup_add_prefix use the current date?

[TarunChillara]

I am facing a similar issue. Can someone respond to this?

[karenzone]

@robbavey @colinsurprenant Will you take a look, please? Could this be a docs issue?

[robbavey]

@karenzone Not a docs issue - the backup_add_prefix setting currently does not support dynamic substitution.

I've added the enhancement label, this looks like a useful feature for this plugin.