search

STOR-i / Changepoints.jl

A Julia package for the detection of multiple changepoints in time series.
Other
32 stars 11 forks source link

BanchmarkLite package is obsolete #33

Open michalkvasnicka opened 2 years ago

michalkvasnicka commented 2 years ago

Could you add some other suitable Benchmark package (supported by the recent Julia Pkg) instead of BenchmarkLite? BechmarkLite is a completely dead project.

See ... perf.jl

michalkvasnicka commented 2 years ago

Any response?

Dom-Owens-UoB commented 2 years ago

Hi Michal,

thanks for pointing this out, for now I'll delete the perf folder and I'l try to update this in the near future. If you have a solution already, feel free to submit a pull request

michalkvasnicka commented 2 years ago

Hi, I am just starting to use Julia, switching from MATLAB. So, at least for now, I am not able to make any suitable submission to your project.

In MATLAB are two functions related to the Changepoint detection by PELT method:

  1. https://www.mathworks.com/help/matlab/ref/ischange.html
  2. https://www.mathworks.com/help/signal/ref/findchangepts.html

I just need to know how fast is your implementation vs MATLAB functions. My experience is, that the original PELT R code (+ internal C functions) by R. Killick is far slower than MATLAB implementation. I am not able to find any reason why is original implementation about 10-100x slower than MATLAB built-in functions.

Did you perform any speed comparisons with original code, or did you make some significant improvements at your Julia code?

Could you help me to evaluate performance of your Julia PELT code? I am using PELT with 'linear' statistics (linear regression segments), threshold = 1.0.

>> tic;[tf,s1,s2] = ischange(signal,'linear','Threshold',1.0);toc
Elapsed time is 0.054550 seconds.

On my PC this task take ~ 0.05sec on signal (86400 samples) ... see attached file (signal.txt ). There are 87 change points see this picture.

Could you send me simple Julia script to read my signal and test your PELT code with proper cost function (linear segments) and measure CPU time?

fairbrot commented 2 years ago

Hi,

You could time a changepoint algorithm in the following way:

using DelimitedFiles
using Changepoints

# Run algorithm once to avoid timing JIT compilation overhead
data = rand(100)
seg_cost = NormalMeanSegment(data, σ)  # Create segment cost function
pelt_cps, pelt_cost = PELT(seg_cost, length(data))   # Run PELT

# Load data
signal = readdlm("signal.txt")
σ = std(signal)

@time begin
    seg_cost = NormalMeanSegment(data, σ)  # Create segment cost function
    pelt_cps, pelt_cost = PELT(seg_cost, length(data))   # Run PELT
end

Note however that this package doesn't currently have a linear segment cost function (please open an issue if you think this would be useful) so it won't be possible to directly compare the two packages for this. I've not done time comparisons but I suspect the R package is a bit faster than our one, since the underlying algorithms are mostly written in C.