search

tsaikd / gogstash

Logstash like, written in golang
MIT License
644 stars 106 forks source link

Can I use dynamic index with elasticsearch output? #134

Open JoHuang opened 4 years ago

JoHuang commented 4 years ago

My data format contains @timestamp field. I would like to send to the index "log-YYYY.MM.DD" according @timestamp. Is this supported? Thanks

tsaikd commented 4 years ago

Currently only provide 2 formats about timestamp:

  1. FormatWithCurrentTime format string with current time, ex: %{+2006-01-02}
  2. FormatWithEventTime format string with event time, ex: %{+@2006-01-02}

You cannot use the timestamp which is not @timestamp field nor now as the index name.

helgeolav commented 3 years ago

Hi

To clarify the question above, to write to an index based on the event time you need something like this in your configuration:

output:
 - type: elastic
    url: ["http://localhost:9200"]
    index: "log-%{+@2006-01-02}"

The syntax on the date format can be found here - scroll down to Layout Options. 2006 is year with four digits, 01 is month with two digits, and 02 is day with two digits.