shenghua-liu
commented
4 years ago why do they need multiple encoder-decoders in Fig 4? what are their purpose? In terms of skip connection, how about randomly choosing a node, and skipping that node, instead of fix length skip. So such randomly choosing skipping will look like drop-out technique. Such random skipping may be good for robustness.
Paper
Outlier Detection for Time Series with Recurrent Autoencoder Ensemble IJCAI'19
Code
https://github.com/tungk/OED
Authors
Task
Outlier Detection for Time Series
Datasets
RNN ensemble framework
Consider Recurrent Skip Connection Networks (RSCNs) (ref Wang and Tian, 2016). Set different L for skip connection and the function:+f'(x_t,h_{t-L})}{2})
For ensemble framework, we need different structures. So we random remove some connection. Specifically, we introduce a sparseness weight vector )
, and make sure each 
at least have one element equal to 1.
The total sparsely-connected RNNs (S-RNNs) can be expressed as
w_t^f+f'(x_t,h_{t-L})w_t^{f'}}{sum(w_t)})
And the figure shows the final structure
Independent Framework
Set different w_t for each autoencoder, and train the autoencoders independently.
Shared Framework
Add a shared layer and concatenate all encoders' codes and use it as init hidden state for all decoder.
Questions
Does anyone know more about different skip connection strategies in RNN?