search

chickenbestlover / RNN-Time-series-Anomaly-Detection

RNN based Time-series Anomaly detector model implemented in Pytorch.
Apache License 2.0
1.17k stars 316 forks source link

how to generate the plots and gif as you posted in the website? #11

Open HongminWu opened 5 years ago

HongminWu commented 5 years ago

Dear there, Thanks for your fantastic work on the time series anomaly detection using RNN techniques. I have several questions for you:

  1. As the title said, how to generate the plots and gif as you posted in the website?
  2. If the proposed method can used to the multivariate time series cases (with multiple sensor channels).
  3. Whether this implementation can be applied online, which means synchronously detecting anomalies after training the models during robot manipulation task?
chickenbestlover commented 5 years ago

Hi HongminWu,

  1. I uploaded a simple python scriptpng2gif.py which generates .gif from a list of .png files.
  2. Yes. In fact, some of the datasets I uploaded two channels (ECG, gesture). I think you can apply this method to data with two or more channels.
  3. This network is small; therefore I think real-time detection during robot manipulation task is possible.
HongminWu commented 5 years ago

ok, thanks for your kind reply. I will check it out.

Best Regards, HongminWu

On Mon, 5 Nov 2018 at 16:53, Jin-Man Park notifications@github.com wrote:

Hi HongminWu,

  1. I uploaded a simple python scriptpng2gif.py which generates .gif from a list of .png files.
  2. Yes. In fact, some of the datasets I uploaded two channels (ECG, gesture). I think you can apply this method to data with two or more channels.
  3. This network is small; therefore I think real-time detection during robot manipulation task is possible.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/chickenbestlover/RNN-Time-series-Anomaly-Detection/issues/11#issuecomment-435798294, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLZeQYI6J3EmcgsWIS0eVYW4jwDEMexks5ur_ySgaJpZM4X7e7p .

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

HongminWu commented 5 years ago

Dear Jin-Man Park,

Good day to you!

So far, I reproduced your Github code followed your detail tutorials. Recently, I want to understand the whole code in a mathematical fashion, but I can't find any your publications about this implementation. Had you published any papers about that? If any, would you kindly share with me via email?

Best regards, Hongmin Wu

On Mon, 5 Nov 2018 at 16:56, Hongmin Wu hongminwu0120@gmail.com wrote:

ok, thanks for your kind reply. I will check it out.

Best Regards, HongminWu

On Mon, 5 Nov 2018 at 16:53, Jin-Man Park notifications@github.com wrote:

Hi HongminWu,

  1. I uploaded a simple python scriptpng2gif.py which generates .gif from a list of .png files.
  2. Yes. In fact, some of the datasets I uploaded two channels (ECG, gesture). I think you can apply this method to data with two or more channels.
  3. This network is small; therefore I think real-time detection during robot manipulation task is possible.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/chickenbestlover/RNN-Time-series-Anomaly-Detection/issues/11#issuecomment-435798294, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLZeQYI6J3EmcgsWIS0eVYW4jwDEMexks5ur_ySgaJpZM4X7e7p .

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

chickenbestlover commented 5 years ago

Hi Hongmin Wu,

This repo is not about my publications, but about implementations of this paper. I studied this paper by implementing the network in the paper, adding some minor changes. Therefore, I don't have any published papers, but the paper I mentioned might be helpful.

Best regards, Jin-Man Park

HongminWu commented 5 years ago

Ok, got it. Thanks a ton.

On Fri, 9 Nov 2018 at 13:57, Jin-Man Park notifications@github.com wrote:

Hi Hongmin Wu,

This repo is not about my publications, but about implementations of this paper https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2015-56.pdf. I studied this paper by implementing the network in the paper, adding some minor changes. Therefore, I don't have any published papers, but the paper I mentioned might be helpful.

Best regards, Jin-Man Park

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/chickenbestlover/RNN-Time-series-Anomaly-Detection/issues/11#issuecomment-437258855, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLZeax0G6EodfrBpVQxswMQ_ech_xoLks5utRlcgaJpZM4X7e7p .

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

HongminWu commented 5 years ago

Dear Jin-Man Park, Recently, I had reproduced your code for anomaly detection using the neural network. However, I got the following questions:

  1. Would you explain more about the implementation of '1-step predictions for target' and 'recursive predictions for target'?
  2. We fit a multivariate normal distribution about the predicted errors for each dimension, respectively, and then we implemented the anomaly detection also for each dimension separately. Am I correct?

Best Regards, Hongmin Wu

On Fri, 9 Nov 2018 at 15:16, Hongmin Wu hongminwu0120@gmail.com wrote:

Ok, got it. Thanks a ton.

On Fri, 9 Nov 2018 at 13:57, Jin-Man Park notifications@github.com wrote:

Hi Hongmin Wu,

This repo is not about my publications, but about implementations of this paper https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2015-56.pdf. I studied this paper by implementing the network in the paper, adding some minor changes. Therefore, I don't have any published papers, but the paper I mentioned might be helpful.

Best regards, Jin-Man Park

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/chickenbestlover/RNN-Time-series-Anomaly-Detection/issues/11#issuecomment-437258855, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLZeax0G6EodfrBpVQxswMQ_ech_xoLks5utRlcgaJpZM4X7e7p .

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​

HongminWu commented 5 years ago

Dear Jin-Man Park, Sorry to interrupt you again. As you said this repo was implemented the idea has proposed in the paper "Long Short-Term Memory Networks for Anomaly Detection in Time Series" by Pankaj Malhotra. I found that in that paper, the error distribution is a multivariate Gaussian Distribution over the error vector that including all the selected d dimensions, where the length of the error vector is d (dimensions) by l (predicted length). However, the function fit_norm_distribution_param() in this repo was designed for each dimension respectively. Am I correct?

For the sake of multivariate time series anomaly detection, would you update those ideas for modeling the error vector across multiple dimensions rather than treat them separately? Specifically, for example, implementing the online multimodal anomaly detection in practical application, we can't trigger an anomaly only consider the perturbations from one specific dimension, we should model it as a unit.

What do you think?

On Fri, 9 Nov 2018 at 13:57, Jin-Man Park notifications@github.com wrote:

Hi Hongmin Wu,

This repo is not about my publications, but about implementations of this paper https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2015-56.pdf. I studied this paper by implementing the network in the paper, adding some minor changes. Therefore, I don't have any published papers, but the paper I mentioned might be helpful.

Best regards, Jin-Man Park

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/chickenbestlover/RNN-Time-series-Anomaly-Detection/issues/11#issuecomment-437258855, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLZeax0G6EodfrBpVQxswMQ_ech_xoLks5utRlcgaJpZM4X7e7p .

-- ​ Hongmin Wu, Ph.D. Candidate Biomimetic and Intelligent Robotics Lab. (BIRL) School of Electromechanical Engineering Guangdong University of Technology Guangzhou, Guangdong, P.R.China 510006 WeChat: Homing20 Facebook: Hongmin Wu Tel:86+18819498204 E-mail:hongminwu0120@gmail.com hongminwu0120@gmail.com URL: https://hongminwu.github.io https://hongminwu.github.io/ http://www.juanrojas.net/research/​