lkorczowski
commented
4 years ago I don't know how you do your resampling
OPTION 1 using panda
https://stackoverflow.com/questions/24920516/downsize-series-containing-boolean-data-with-resample
with something like
ds = ts.resample('0.25ms', how=np.any)
if you want to say that any positive within 250ms
- pro: seems to work fine to downsample, input whatever merge function we like (np.any, np.all, etc.)
- con: need to recompute timestamp, didn't find a way to upsample and dealing with nan
OPTION 2 using scipy
e.g. scipy.signal.resample_poly
maybe overkill but it should we if the up or downsampling factor is a multiple of the number of epochs (but usually, it won't). Maybe worse a try.
- pro: ?
- con: overkill and maybe too slow. need to add a logic function after (e.g. >0) because the output won't be boolean.
OPTION 3: simple moving average (or reverse moving average)
- pro: would work nicely
- con: would probably be very slow because ugly for loop ^^ would be error prone because we will always have a case we it doesn't work. Still require a logic
When dealing with valid_labels merges, here is one important pytesting I've done:
dealing with tricky case 1:
dealing with tricky case 2:
Unhappily the algo returns [True, True] for first test and [True False] for second, so the first assert fails. There should be a "domination" of false over True that makes both cases go in the same direction.
Again, it's just a singular case as in practise we never go in the direction of downsampling.