search

ait-aecid / logdata-anomaly-miner

This tool parses log data and allows to define analysis pipelines for anomaly detection. It was designed to run the analysis with limited resources and lowest possible permissions to make it suitable for production server use.
GNU General Public License v3.0
79 stars 24 forks source link

Z in timestamp #1306

Closed landauermax closed 4 months ago

landauermax commented 7 months ago

A common timestamp format is "2024-03-21T20:46:06.955Z", which has the "Z" in the end, representing UTC. I do not think that this is currently supported to be parsed with our %z format. Also, this sample should be included in the aminer documentation as it is very common.

ernstleierzopf commented 6 months ago

This can easily be represented using a format such as "%Y-%m-%dT%H:%M:%S.%fZ". However, I extended the %z format to also include parsing Z specifically to improve usability.

landauermax commented 6 months ago

Yes, the format can be adapted, but then the Z is not read as the timezone. This means that the timestamp of the log atom is not correct, which can be a problem when the timestamp is further used (e.g. when anomalies are processed and visualized). It is therefore important that Z is read as UTC.

ernstleierzopf commented 6 months ago

Yes, the format can be adapted, but then the Z is not read as the timezone. This means that the timestamp of the log atom is not correct, which can be a problem when the timestamp is further used (e.g. when anomalies are processed and visualized). It is therefore important that Z is read as UTC.

Added the change to include it in the %z parameter in #1316. Generally both solutions should/will work, because UTC is the default timezone of the DTME.