florisvb / PyNumDiff

Methods for numerical differentiation of noisy data in python
Other
111 stars 16 forks source link

TVR iterative velocity crashing when scale = large #31

Closed sameervk closed 1 year ago

sameervk commented 2 years ago

Hello,

I am trying to use different methods to smoothen time series data (about ~52000 time points). When using options cg_maxiter: 1 or None with 'scale': large, the kernel crashes. Tried multiple times but same issue occurs. Other parameters are iterations = 1 and gamma = 0.001, whilst dt = 1.

When I don't specify the options argument, it runs fine but I believe here it uses 'scale: small'. Wanted to test when scale = large.

Thank you

florisvb commented 2 years ago

This is with the total_variation_regularization.iterative_velocity method? Does the kernel crash with a smaller number of time points? 52,000 is a lot of data. If it works with smaller datasets you can run several smaller timeseries through and then blend. See jerk_sliding for an example on how to implement the blending.

On Tue, May 3, 2022 at 2:33 AM Sameer Kesava @.***> wrote:

Hello,

I am trying to use different methods to smoothen time series data (about ~52000 time points). When using options cg_maxiter: 1 or None with 'scale': large, the kernel crashes. Tried multiple times but same issue occurs. Other parameters are iterations = 1 and gamma = 0.001, whilst dt = 1.

When I don't specify the options argument, it runs fine but I believe here it uses 'scale: small'. Wanted to test when scale = large.

Thank you

— Reply to this email directly, view it on GitHub https://github.com/florisvb/PyNumDiff/issues/31, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB4EPGZB32OXLFRG75XJ7DVIDXFVANCNFSM5U6L6TSA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Floris van Breugel | http://www.florisvanbreugel.com Assistant Professor of Mechanical Engineering & Graduate Program for Neuroscience University of Nevada, Reno

Wildlife and Landscape Photography Galleries: http://www.ArtInNaturePhotography.com/ Blog: http://www.ArtInNaturePhotography.com/wordpress/

sameervk commented 2 years ago

Yes, that's correct, this is with the total_variation_regularization.iterative_velocity method. I didn't try with smaller number but I will give this and the other suggestions you made a shot. Thank you.

florisvb commented 1 year ago

Closing this, in general the TVR methods are not recommended for very large datasets.