not working with metricbeat data (?)

[hokiegeek2]

Hi @eramuk,

I most likely have something misconfigured, but for whatever reason, I think this plugin is not working with the metricbeat logstream I am testing. Here's my configuration:

<label @metrics>
  <filter *beat>
    @type record_transformer
    enable_ruby true
    renew_record true
    <record>
      timestamp ${record["@timestamp"]}
      metricset-rtt ${record["metricset"]["rtt"]}
      route metricset
    </record>
  </filter>
  <filter *beat>
    @type time_sampling
    unit ${tag}, timestamp, route
    interval 10
  </filter>
  <match *beat>
    @type copy
   <store>
      @type file
      @id output_file
      path /opt/data/metricbeats
    </store>
   </match>
</label>

This configuration grabs the rtt JSON field from metricbeat and streams it independent of the remaining JSON-formatted message; this part definitely works. Such messages are being emitted from metricbeat anywhere between 100 ms to 3 seconds. I need for a message to be sampled and sent out every 10 seconds. I need this for a streaming time series anomaly detection use case.

I may have something misconfigured or it may be that I am misunderstanding the use case(s) this plugin is intended to address. Any guidance is definitely appreciated.

Thanks

--John

[eramuk]

Please try remove "timestamp" from the "unit" key.

    unit ${tag}, route

I tried your configuration (removed timestamp) and metricbeat, I got following results. The one event is sent every 10 seconds.

2018-10-01T22:39:46+09:00       beat    {"timestamp":"2018-10-01T13:39:46.746Z","metricset-rtt":"106","route":"metricset"}
2018-10-01T22:39:56+09:00       beat    {"timestamp":"2018-10-01T13:39:56.749Z","metricset-rtt":"133","route":"metricset"}
2018-10-01T22:40:06+09:00       beat    {"timestamp":"2018-10-01T13:40:06.746Z","metricset-rtt":"75","route":"metricset"}

If this is not what you want, please send additional information.

[hokiegeek2]

Perfect, thank you! This is exactly what I need this to do, thanks @eramuk!