search

gaiusyu / Brain

Brain: Log Parsing with Bidirectional Parallel Tree
Apache License 2.0
27 stars 6 forks source link

Group_accuracy #5

Open Tasneem91 opened 10 months ago

Tasneem91 commented 10 months ago

when I run your code, the group accuracy result comes equal to the accuracy result which is not true, and it does not match the results in your paper. would you please give help in this

gaiusyu commented 10 months ago

We have updated some parameters in the code, so the results obtained by running the code are slightly different from the results in the paper, but the average GA should be around 0.98. If you still have questions, please do not hesitate to contact me.

Tasneem91 commented 10 months ago

ga ga I just run the code, and print the accuracy, I just have a problem why accuracy equal to group accuracy, the y are different in the definition and the meaning. I need to know the correct way to find GA @gaiusyu would you please help me with this issue

gaiusyu commented 10 months ago

"Accuracy" and "F1 score" are obtained by calling the evaluate function in Logpai. In fact, "Accuracy" in LogPai is GA. I re-implemented a GA function to get "Group Accuracy". The difference between the calculation methods of my function and the function in LogPai is the criterion for judging whether the log belongs to the same group. The function in LogPai looks at the log template ID of the log, such as "E12", while my function looks at the specific template. , such as "session closed for user <*>". Previously, LogPai's annotation was incorrect. some IDs belonging to different templates were annotated as the same, which resulted in LogPai's function calculation "Accuracy" being incorrect. Generally speaking, "Accuarcy" and "Group accuracy" are both GA.