search

KnowledgeDiscovery / rca_baselines

Code for "LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis" paper
https://lemma-rca.github.io/
Other
10 stars 2 forks source link

What does the Product Review dataset mean when you have achieved 100% perfect performance? #3

Closed FeiGSSS closed 2 months ago

FeiGSSS commented 2 months ago

Hello authors,

Thanks for your posts! I'm new to this area and looking for a robust dataset to start my research. I noticed that your paper reported that the MULAN method achieved 100% scores for every metric in the Product Review dataset, I'm wondering then what is the point of proposing this dataset as it seems to be solved now?

KnowledgeDiscovery commented 2 months ago

Hi Fei,

Thank you for your interest in our work!

Even though MULAN has achieved high performance on the Product Review dataset, it remains valuable for several reasons:

  1. Benchmarking and Comparisons: It provides a standard for comparing new methods and evaluating their performance against a well-established benchmark.
  2. Continued Research Opportunities: The dataset is useful for exploring new techniques, testing different configurations, or adapting to new contexts.
  3. Real-World Applications: High performance on this dataset doesn’t guarantee similar results in more complex, real-world scenarios.
  4. Evolution of Techniques: It allows for tracking advancements in the field as new methods and improvements are developed.

Additionally, we would also like to highlight that our LEMMA datasets include more challenging scenarios like Cloud Computing: MULAN's performance decreases on this more complex dataset. This highlights ongoing research opportunities and the need for advancements in handling such datasets.