sintel-dev / Orion

A machine learning library for detecting anomalies in signals.
https://sintel.dev/Orion/
MIT License
993 stars 159 forks source link

TimeStamp Number intervals Less than 109 cause error #553

Open NIckyVNYC opened 3 weeks ago

NIckyVNYC commented 3 weeks ago

Description

I am training a model using the AER pipeline. Previously using TadGan, I successfully trained a model using astrophysics data. For the timestamp, to make the time consistent between different files I was using, I substituted the timestamp to be 1, 2, 3, 4, 5, ect. With AER, it doesn't work and I get this error: ValueError: Axis 2 is outside the dimensions of X (1). However, if I use any interval for the time of 109 or above, it works, like 109, 218, etc. If I use lets say 108 as the timestamp interval, I get this error.

ValueError: Training data contains 1 samples, which is not sufficient to split it into a validation and training set as specified by validation_split=0.2. Either provide more data, or a different value for the validation_split argument.

I guess for my project I could use intervals for the timestamp of 109 or above which appear to work. But I would rather just use what worked with TadGan, intervals of 1, such as 1, 2, 3, etc for the timestamp. With AER, what do I have to change so that timestamp intervals are 1, 2, 3, etc,

This is my set up code for Orion-ML

hyperparameters = { 'orion.primitives.aer.AER#1': { 'epochs': 5, 'verbose': True, 'input_shape': [100, 4] } }

orion = Orion( pipeline='aer', hyperparameters=hyperparameters,

)

sarahmish commented 2 weeks ago

Hi @NIckyVNYC, thank you for using Orion!

How long is your time series?

Also, to confirm, in addition to changing the AER hyperparameters, you are changing the interval setting as well? I imagine your code will look something like this

hyperparameters = {
    'mlstars.custom.timeseries_preprocessing.time_segments_aggregate#1': {
        'interval': 1
    },
    'orion.primitives.aer.AER#1': {
        'epochs': 5,
        'verbose': True,
        'input_shape': [100, 4]
    }
}

orion = Orion(
    pipeline='aer',
    hyperparameters=hyperparameters,
)