search

deepcharles / ruptures

ruptures: change point detection in Python
BSD 2-Clause "Simplified" License
1.59k stars 162 forks source link

Detection fails when dealing data has interval with 0 variance #196

Closed chen-yutong closed 2 years ago

chen-yutong commented 3 years ago

Hi When I try the change point detection on some real-world stock transaction count data, I found the algorithm fails to output change points, even though the configuration of change point is quite obvious. I think the cost function causes the issue, because my 1-d input data has 0 variance when the stock market is closed. I use normal cost in costnormal.py, and the val becomes negative infinite under this circumstance. As a result, the change of cost value will become discontinuous. _, val = slogdet(cov) To justify my hypothesis, I tried to add a small bias on the determinant of the covariance matrix can fix the problem. I change this line into _, val = slogdet(cov + 1) for my 1-d data, and the change point detector outputs perfect result. This change still keeps the monotonicity of the cost function. Hence, I think the problem comes from the cost function. Could you make some modification to handle this issue when det(cov) = 0?

deepcharles commented 3 years ago

Thanks for your interest in ruptures.

Indeed, we should add a small bias to make the computation more stable.

deepcharles commented 2 years ago

Solved in #198

If you install from source, or wait for the next version (i.e. 1.1.5), your code should run as expected.

Feel free to reopen if that does not solve your problem.

chen-yutong commented 2 years ago

Thanks. Your work on change point detection really help me a lot.