sintel-dev / Orion

A machine learning library for detecting anomalies in signals.
https://sintel.dev/Orion/
MIT License
1.03k stars 159 forks source link

How to see reconstructed time serie? #432

Closed mohammadx0098 closed 11 months ago

mohammadx0098 commented 1 year ago

Hi

1. I want to use tadgan_without_gpu_dropout and have the visualized option.

How can I do that?

I am training my model and I think my problem is related to ability of my model to reconstruct the time serie.

2. When I'm usiing orion-ml in my windows local machine I have problem with mathplotlib and I think it's because of the conflict between your package and matplotlib in numpy or pandas version.

Which version of matplotlib is compatible with your package?

mohammadx0098 commented 1 year ago

After learning with these hyperparameters :

hyperparameters["epochs"] = 5 hyperparameters["input_shape"] = (100, 1) # based on the window size hyperparameters["optimizer"] = "keras.optimizers.Adam" hyperparameters["learning_rate"] = 0.0005 hyperparameters["latent_dim"] = 20 hyperparameters["batch_size"] = 64 the model learned the time serie but why th amplitudes are not the same?

IMG_20230716_001828_850

Now just plot the reconstructed signal :

IMG_20230716_001830_530

sarahmish commented 1 year ago

Hi @mohammadx0098, thanks for using Orion!

  1. To do that, you need to add a visualization option to the pipeline json. please add the following block to the tadgan_without_gpu_dropout pipeline after the output_names block.

    "outputs": {
    "default": [
        {
            "name": "events",
            "variable": "orion.primitives.timeseries_anomalies.find_anomalies#1.y"
        }
    ],
    "visualization": [
        {
            "name": "generated_timeseries",
            "variable": "orion.primitives.tadgan.score_anomalies#1.predictions"
        }
    ]
    }
  2. Here is matplotlib version that I use

    matplotlib==3.5.3
    matplotlib-inline==0.1.6

    you can also let pip resolve this compatibility by installing both packages together using pip install orion-ml matplotlib

  3. There are extreme values in the original time series that are probably affecting the learning process and shifting your reconstructed time series.

Hope this helps, let me know if you have any further questions!

mohammadx0098 commented 1 year ago

@sarahmish Thanks.

  1. Why generated_timeseries has morr than 1 result . Which one should I plot to see the reconstructed signal?

3.I train the model witjout extreme values. The shift is solved but I have still have the problem with amplitute of the signal with is not the same as original one. I will send the results

sarahmish commented 1 year ago

Hi @mohammadx0098!

The generated_timeseries contains 5 dimensions representing the minimum, 25th, 50th, 75th, and maximum values respectively. To plot the results, I made a quick snippet here to visualize the generated time series

import matplotlib.pyplot as plt

signal = viz['generated_timeseries']
signal = signal.squeeze(1)

x = len(signal)
median = signal[:, 2]
y1 = signal[:, 1]
y2 = signal[:, 3]

plt.plot(median)
plt.fill_between(range(x), y1, y2, alpha=0.3)
plt.show()