how to efficiently predict chaotic serialized data , like lorenz ?

[JingliangGao]

Hi, Referring to the examples you provide, I have successsfully write the code to predict Lorenz serialized data, though the result is terrible. Um ... I still don't understand some details as follows.

---

1. Why we set 'washout ' this variable ? For example, when we set washout be 10 and try to use trx (length 30) to predict trY, we may get an output with length 20 (30-10= 20). BTW, I have read your comments in echo_state_network.py, but still don't understand the reason.

2. How we recogize 'Deep' this concept ? I have had a glance at the paper of Deep-ESN, and it seems that muti-reserviors have been connected to make this structure be deeper. I have read your code , but can not see this idea.

   ---

I am looking for your reply. :) @stefanonardo

[stefanonardo]

Hi JingliangGao,

1. We set the first hidden state to zero, therefore the initial internal states of the reservoir are really biased from this initialization and do not provide useful information. For this reason, it is a common practice to discard an arbitrary number of steps. See ""A Practical Guide to Applying Echo State Networks" for a better explaination.

2. I do not entirely understand your question. Do you want to see where is the "deep" feature inside the code? Here, the implementation of the deep ESN is the same as the PyTorch's implementation of deep RNN. The difference is that we collect the output of each layer (reservoir), not just the last one, since, at the end of the training, we need the state of each reservoir.

Regarding the Lorenz timeseries, if you can share code and dataset, or at least the dataset, I could tell you if something is wrong.

[JingliangGao]

Hi Stefanonardo, Thanks for your prompt reply and help me solve the Q1 that I mentioned above.

1. Yeah, I want to see where is the "deep" feature inside the code, but find nowhere. Specially, to make the structure be deeper, we can collect the output of each layer (reservoir), so where can this be reflected in your code?
2. For Lorenz serialized data prediction, I will send the dataset to you mailbox stefano_nardo@msn.com. BTW, I use the time delay method to train and predict Lorenz serialized data, but the loss does not converge. Do you have any great idea?

It will be appreciate if you provide more detials. Thanks :) @stefanonardo

[stefanonardo]

1. If you are asking how to make a deep ESN, just set num_layers to a value greater than one when you create an ESN. If you want to see the code that makes it work... You should take a look at function StackedRNN in reservoir.py. The logic is all there more or less.
2. Do you have any reference to time delay method? I do not think to know what it is. Anyway, what is Lorenz serialized data? Is it just the Lorenz timeseries in a serialized format or what?

[JingliangGao]

You are right. Lorenz serialized data is the Lorenz timeseries data. I will sent you an example to your mailbox soon.