Kibana showing the wrong date

[vichargrave]

I've raised this issue before and it has been closed, but no explanation of how to deal with it has been given. Kibana shows the wrong date that it isn Elasticsearch.

I realize this is because the browser time zone is different than the UTC date. The point is this is still a bug, because Kibana should use the date in Elasticsearch not the browser. What is the workaround for this issue?

Please do not close this issue until you have provided an adequate answer.

[spalger]

this is still a bug, because Kibana should use the date in Elasticsearch not the browser

What do you mean by "the date in Elasticsearch"? The elasticsearch docs cover how elasticsearch sees all dates as UTC. Do you think that we should always show dates in UTC? UTC is a great format for computers, but not for humans, so we localize times to the timezone you have told your computer to use.

If you do want all dates and times to be in UTC then I suggest that you change your computer's time settings.

We realize that the ability to change the timezone which Kibana localizes to is something that people want to control. That is why tickets like https://github.com/elastic/kibana/issues/1600 exist. When implemented this ticket will allow you to perceive your data as if you were in New York, Tokyo, or straddling the prime meridian, whatever makes sense for your data.

Here are some workarounds you can implement today:

• change 2015-07-30 to 2015-07-30T00:00:00.000-0700 so elasticsearch knows that your events occurred at midnight in your timezone.
• store the actual time that the event occurred and receive accurate information relative to how you have told your computer's clock to behave

Please let me know if this answer is adequate to your expectations.

[vichargrave]

Thanks for that explanation and the ticket reference. You make a good point. I guess what I want ultimately is to express the dates and times in UTC. I will try your suggestions, which I think solve my problem for the time being, and I'll look for the feature in ticket 1600.

[vichargrave]

Ok I just indexed some data and used the UTC date at the time to set the date timestamp. The UTC date of running the script was 2015-08-25 20:24:57 and my local time was 2015-08-25 13:24:57. Here is what 1 of the documents looks like:

Notice the date is 2015-08-24 which is not the correct UTC time for my local system. Unless I'm missing something, this date should read 2015-08-25 even in my time zone (-07). This date looks like a day early which is the problem with which I have been struggling.

[spalger]

What does the raw JSON show the value is?

[vichargrave]

The date in the JSON is 2015-08-25. See below:

[spalger]

Okay, well in order for elasticsearch to know that this date is supposed to be 2015-08-25T20:24:57Z (the time that the script executed, in UTC timezone) you must index the entire timestamp. By only indexing 2015-08-25 elasticsearch assumes you want that field to represent 2015-08-25T00:00:00Z

[spalger]

Once you have indexed the entire timestamp you can tell Kibana to format the timestamp as just a date using a field formatter.

[vichargrave]

I put entered the date with time zone information. Here is what it looks like:

This is the raw JSON. Notice that Elasticsearch shows the time in my local zone. If Elasticsearch treats all dates as UTC then why is this date showing in the -0700 time zone?

This is the problem I've had all along and why I said Kibana should just take the time from Elasticsearch then return it without conversion. If I put the date and time in a date field with time zone information, I want it to be shown in Kibana the way it is in Elasticsearch without any conversion in any time zone. That way my dates will be correct in any time zone. Hopefully future versions of Kibana will have a "don't convert the dates" mode.

[ghost]

If you convert the timestamps you write in your JSON documents to UTC before indexing them, your problem would be solved. We are doing that here - we handle all of our timestamps as UTC in our applications and logs and index them into ES with the format "yyyy-mm-ddThh:mm:ssZ". Kibana localises them properly to the Tokyo timezone when we view our documents and we see the expected dates/times.

I strongly recommend indexing in UTC rather than trying to index with a timezone.

[vichargrave]

The problem is I don't want Kibana to localize the time. We have other timezones that use the interface and each will get different time results based on the localization of the browsers. I want the data back from ES unaltered.

[spalger]

@michaelredfox I think what @vichargrave is asking for now is for us to read the "-7000" in the date field and render the time in that timezone when displaying it rather than the local timezone. It's an interesting idea, but could lead to really strange results if each document has a different timezone listed.

[spalger]

@vichargrave Have you tried formatting the date as a string?

[vichargrave]

Why would I want to do that? Then I can't use date ranges properly. Also with regard to your previous comment, dates will frequently span multiple time zones. If Kibana converts them all to local time zone it is an utter mess. Now I should say that I'm indexing time series data in my various applications which is unlike a more standard search where you have indices that house data that is not so time dependent. But when it comes to more SIEM style application it is critical that time is not adjusted by Kibana.

[spalger]

Sorry I meant format the date for display as a string (using kibana field formatters)

[vichargrave]

Ok, no I'm not sure how that works.

[spalger]

You can set a format each field in an index pattern, and your options are things like "number", "date", "string", etc. The default format for date fields is the "date" field format, which is what is converting the time to local time. By changing this format to "string", the value that is stored in the elasticsearch document will be displayed in plain text.

You can edit these formats by going to settings > indices, click on your index pattern on the left, and then click the little edit button to the right of the field you want to change. Alternatively, us can use this url pattern:

http://localhost:5601/#/settings/indices/{{your index name}}/field/{{ your field name }}

[ghost]

@spalger Ah I see. Yes, it would lead to strange results if that were the case. @vichargrave I understand your issue now.

Related to this issue, I support the idea of optionally having Kibana show timestamps in UTC (or a timezone of your choice - not sure how you would handle DST nicely though) rather than just using your browser timezone. A setting in config.yml perhaps? Sometimes it is useful to be able to view logs with UTC timestamps when comparing events in Kibana to those in other (not stored in ElasticSearch) logs that are written in UTC without having to do the mental conversion.

Perhaps even have it as a setting in the dashboard, so that it is possible to create a localised dashboard or UTC dashboard? I think that may make things much more complicated though.

However I feel that the default should be as it is right now - converting the timestamps to your browser timezone. It was pleasant to see Kibana formatting my logs in a form that is comfortable to use when I first installed it.

[spalger]

@michaelredfox glad you agree that the current behavior is probably the correct default. I'm going to close this as a duplicate of #1600, but I'll keep an eye on this ticket, let me know if you have further questions @vichargrave

[aliemam]

@spalger im sorry but i realy cant understand why kibana even convert date in first place. options like set time zone is for current time not for for changing data. any fields should be manipulated by owner of data not kibana. it just an interface. if the person indexes data in elastic was me so it should be me to manipulate data. kibana is great visualizer and i really appreciate that. but please manipulate some data user indexed is not good. i think timezone is great option but just to change current time not some date specially as some data.

[bryhoang]

I also do not understand the rationale behind localizing all date/times to your browser. I am pulling data from multiple data centers from all over the world, so when the date/time is converted to the local browser times that confuses my users. Because, when they lookup the value in the database and compare that to what Kibana is showing it does not match. Now, I have to tell them to convert the time +/- their timezone? There really needs to be a way to turn OFF this "feature." I completely agree with vichargrave, the date/time is inputted into elasticsearch exactly how my original data has it, I can even query the date/time through Kibana and get the original date/time. So, why on earth should Kibana change that while viewing it in the browser?

This is causing a huge mess for me, because my users want to see the original date/time logged in the source database, NOT what Kibana thinks makes sense. So, either I figure out how to trick Kibana into doing the correct thing or use a different presentation layer.