Investigate LSTNet

[zer0n]

and integrate it into this library if it works well.

Also compare this with the Tensor-Train RNNs paper.

[bischtob]

Here is also the Tensor-Train repo from the author: https://github.com/yuqirose/tensor_train_RNN

[austingross]

@bischtob helpfully pointed out that the Tensor-Train RNNs paper has open reviews suggesting other techniques, specifically Generating Sequences With Recurrent Neural Networks, Grid Long Short-Term Memory and Hierarchical Multiscale Recurrent Neural Networks.

I'll investigate these as well. I'd like to come up with a framework for comparing our method to these.

[zer0n]

Keep in mind that references in paper or suggested papers in peer reviews are more often than not lightly related or solving corner cases or just too increase citation count of authors/reviewers rather than an alternative method that is mature and generally applicable.

I suggest that we don't get too much distracted by these noises.

On Mon, Apr 9, 2018, 9:53 AM austingross notifications@github.com wrote:

@bischtob https://github.com/bischtob helpfully pointed out that the Tensor-Train RNNs paper https://arxiv.org/abs/1711.00073 has open reviews https://openreview.net/forum?id=HJJ0w--0W suggesting other techniques, specifically Generating Sequences With Recurrent Neural Networks https://github.com/snowkylin/rnn-handwriting-generation, Grid Long Short-Term Memory https://github.com/coreylynch/grid-lstm/ and Hierarchical Multiscale Recurrent Neural Networks https://github.com/n-s-f/hierarchical-rnn.

I'll investigate these as well. I'd like to come up with a framework for comparing our method to these.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MSRDL/Deep4Cast/issues/13#issuecomment-379820054, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6gZy0wxoI6vgd81CE0wJfpmmF0n7l9ks5tm5INgaJpZM4S38FT .

--

Sent while mobile

[austingross]

@zer0n understood. We'll start with LSTNet and go from there. I've been working on mapping the pytorch functionality that it was written in to keras.

[zer0n]

You should first test whether it's significantly better than our current baseline or tensor-train on a few datasets before doing the PyTorch -> Keras conversion.

[behdad84]

I did a comparison of the Tensor-Train-RNN (found in https://github.com/yuqirose/tensor_train_RNN) with the RNNForecaster and CNNForecaster methods (already exists in this code-base). On the the 26 dimensional dataset (data/sample3.csv), I did not observe any improvement. Since both RNNForecaster and CNNForecaster seem to currently only support one-step-ahead prediction, I also ran the tensor-Train-RNN for predicting next immediate steps.
I tried the method with different values of (lag, hidden size and learning rate) parameters but did not observe any improvement. The measure was the mean euclidean distance of the predictions and actual. The notable difference is the computational cost. Tensor-Train-RNN runs significantly slower than the other two methods. Tensor-Train-RNN paper claims that the method is able to produce better results for more complex dynamics (higher order time dependencies). The implementation is also based on the sequence-to-sequence encoder-decoder which is claimed to generate better results for long-term forecasting. I think a more fair comparison would be to have a dataset with these properties.

If you know of any other good dataset that we can run this on please let me know.

[zer0n]

sample3.csv is just a toy dataset, for demo purpose. We should add a couple more datasets to the repo for more comparisons. @austingross , I think you're on it?

[austingross]

@behdad84 I messaged you providing some datasets that we've used in benchmarking. I'll follow up with you separately.

@zer0n yes, I'll be responsible for getting standard benchmark data sets in.