IntelligentDDS / Nezha

The implementation of multimodal observability data root cause analysis approach Nezha in FSE 2023
MIT License
21 stars 5 forks source link

Wrong time stamp in metrics file #8

Open mmantyla opened 8 months ago

mmantyla commented 8 months ago

This file has incorrect timestamps. https://github.com/IntelligentDDS/Nezha/blob/main/construct_data/2022-08-22/metric/adservice-5f6585d649-fnmft_metric.csv They all start with 18 when they should start with 16, e.g. 1861140279 when it should be 1661140279.

There could be other files as well. I am building a data loader for Nezha to https://github.com/EvoTestOps/LogLead so I just did raw string replacement and it did not check all files.

mmantyla commented 8 months ago

Also this file https://github.com/IntelligentDDS/Nezha/blob/main/rca_data/2023-01-29/log/09_34_log.csv Has time stamps like -6795364578871340000 when it should be 1679... Again there might be others.

yuxiaoba commented 8 months ago

Hi, Professor Mäntylä, thanks for pointing out the timestamp issue, I'll fix it as soon as I can!

mmantyla commented 8 months ago

Hi, I found that also one 2023-01-29-fault_list.json had issues. The human readable and epoch time did not match. I am not sure if it is actually worthwhile spending your time on. I worked around by using the human readable time format. If I parse that then so far nor problems. You might want to add a note about that to the front. If one tries to use epoch time there will be plenty of issues.

daixixiwang commented 1 week ago

Hi, I also found time stamp error in trace files. For example, in file Nezha/rca_data/2023-01-30/trace/11_51_trace.csv,

TraceID,SpanID,ParentID,PodName,OperationName,StartTimeUnixNano,EndTimeUnixNano,Duration
aa00265193e2a75bdbb109b5a286b3a3,dd8e061dd1813199,root,ts-gateway-service-6f6cfc45b-d9pnv,/*,1675079501299000000,1675079501682364930,383364

Duration is 383364 is equal to 106 hour, 29 minutes, and 24 seconds, which is unrealistic for a trace Duration.

So, will these data be fixed? @yuxiaoba

And can you open your fixed data, if possible ? @mmantyla As I have worked on this data for quite a lot experiments, but found this problem at this time period.

mmantyla commented 1 week ago

Hi, I found only the epoch time problematic. Human readable time was correct as far as I can tell.