Daily index date is based on local date rather than UTC

Hi,

Because the reporter assumes that the date used in the index is the local date, rather than UTC (which ES presumes) we find that our metrics are being split over two indices (we are storing metrics in daily indices and are in CET (GMT+1) Timezone).

From the code I see that the reporter derives the index name from a Date object, and the SimpleDateFormat uses the local Timezone by default.
This means that at midnight Jan 25 CET the reporter send metrics to a new Jan 25 index, whereas the metrics are still timestamped Jan 24 UTC.

Setting the Timezone of the SimpleDateFormat resolves the problem. The following snippet hopefully helps to explain.

  long timestamp = 1485298800; // Wed Jan 25 00:00:00 CET 2017 / Wed Jan 24 23:00:00 UTC 2017

  System.out.println("timestamp = "+timestamp*1000);
  // timestamp = 1485298800000

  System.out.println("Date = " + new Date(timestamp*1000));
  // Date = Wed Jan 25 00:00:00 CET 2017

  String currentIndexName = "metrics";
  SimpleDateFormat indexDateFormat = new SimpleDateFormat("yyyy.MM.dd");
  currentIndexName += "-" + indexDateFormat.format(new Date(timestamp * 1000));
  System.out.println("currentIndexName (CET) = " + currentIndexName);
  // currentIndexName (CET) = metrics-2017.01.25

  currentIndexName = "metrics";
  indexDateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
  currentIndexName += "-" + indexDateFormat.format(new Date(timestamp * 1000));
  System.out.println("currentIndexName (UTC) = " + currentIndexName);
  // currentIndexName (UTC) = metrics-2017.01.24

To clarify why this is a problem for us (besides the ES indexes being UTC oriented), we run an aggregator on each index and then can simply delete the entire index. As it stands, an hours worth of values from the next day are missing, and are then bundled into the subsequent run. Furthermore, the logic in this reporter does not match how logstash allocates documents to indices, i.e. 00:00->23:59 UTC in one index.

Changing the behaviour of the reporter now might not be desirable, but adding the option to choose between UTC index dates or local would be very useful, or at least mentioning it in the documentation might help, in case it trips someone else up.

I'll happily create a Pull Request if you wish.

elastic / elasticsearch-metrics-reporter-java

Daily index date is based on local date rather than UTC #52