search

logpai / loglizer

A machine learning toolkit for log-based anomaly detection [ISSRE'16]
MIT License
1.27k stars 423 forks source link

Invariant miner taking hours to run #64

Closed Rufaida94 closed 4 years ago

Rufaida94 commented 4 years ago

Thank you for the invariant miner, it is a great tool for anomaly detection. However, I have one issue with it:

when running it with some log files from different sizes (it works with some log files and does not work with others and this is not dependent on the size of the log file) sometimes it gets stuck ( I leave it running for hours and still no result).

it seems to be stuck in this position: ====== Model summary ====== Invariant space dimension: 17

so it estimates the Invariant space dimension but fails to produce the actual invariant.

when I try to deubg I find that this is the point in the code that is taking so long:

Traceback (most recent call last): File "InvariantsMiner_demo_without_labels.py", line 27, in model.fit(x_train) File "../loglizer/models/InvariantsMiner.py", line 44, in fit self._invariants_search(X, invar_dim) File "../loglizer/models/InvariantsMiner.py", line 132, in _invariants_search joined_item_list = self._join_set(item_list, length) # generate new invariant candidates File "../loglizer/models/InvariantsMiner.py", line 264, in _join_set if joined not in return_list:

Any idea why? and how can I fix this?