What does GRU mean for this crate?

[dbsxdbsx]

I know GRU is a variant of RNN in the context of neural network of machine learning. And in the original NEAT paper, the author did mention a kind of RNN, with which the game double pole is solved----the RNN here is not what we refer to in the context of machine learning, right?

But what does GRU mean in the context of NEAT for this crate? Does it mean that NEAT could generate GRU structure (like in this paper), and with which could solve non-Markov decision problem?

[sakex]

Hey, GRU stands for Gated Recurrent Units which I believe is the same concept as the ones explored in your paper.

In this crate, connections between neurons can evolve to have GRU gates or not.

[dbsxdbsx]

Hey, GRU stands for Gated Recurrent Units which I believe is the same concept as the ones explored in your paper.

In this crate, connections between neurons can evolve to have GRU gates or not.

Thansk your feedback. So what paper you referred to when you building GRU for this crate? I do want to know more about RNN+NEAT.

[sakex]

This paper:

https://scholar.google.ch/scholar_url?url=https://arxiv.org/pdf/1412.3555)&hl=en&sa=X&ei=xx1tY5eeG6mSy9YPyq-I8Ak&scisig=AAGBfm1QLoqDwmIOWXtWsBrcUTdX3K7r9g&oi=scholarr

The implementation here is a bit convoluted because of the fact that we represent NNs as graphs (sparse NNs) .

Basically a connection will keep a hidden state if it is a GRU connection and will reset and update according to the paper. Weights are learned with the Genetic Algorithm.

[dbsxdbsx]

This paper:

https://scholar.google.ch/scholar_url?url=https://arxiv.org/pdf/1412.3555)&hl=en&sa=X&ei=xx1tY5eeG6mSy9YPyq-I8Ak&scisig=AAGBfm1QLoqDwmIOWXtWsBrcUTdX3K7r9g&oi=scholarr

The implementation here is a bit convoluted because of the fact that we represent NNs as graphs (sparse NNs) .

Basically a connection will keep a hidden state if it is a GRU connection and will reset and update according to the paper. Weights are learned with the Genetic Algorithm.

The paper you mentioned is trying to compare advantages over different RNN gate units. I have no doubt that both LSTM and GRU (or other gate-based RNN units) could solve supervise problems.

What I wonder is if these RNN units could be used to solve non-Markov decision problems. Actually, when seeing the word RNN in the original NEAT paper, I thought the answer is yes. But after digging deep, noticing that the RNN unit is just s simple hidden recurrent connection inside the generated network, since there is no hidden input for each decision state, the answer seems to be no?

If I understand correctly, neat-gru-rust is just replacing the original NEAT-RNN with the gru gate unit?

[sakex]

Yes exactly. I made this crate as part of an automated trading strategy (non Markov) and then decided to open source it .

[dbsxdbsx]

Yes exactly. I made this crate as part of an automated trading strategy (non Markov) and then decided to open source it .

Actually, I am also interested in putting NEAT or RL for trading. But here there are 2 basic things I need to make clear first.

1.How does neat-gru(your crate) behave when choosing an action in a no Markov env? In detail, is there a "hidden state" that is produced by gru unit , and then the hidden state serves as an extra input for the next env state? If this is the case, a more detailed question: does the hidden state work as a DIRECT input for the gru-unit node?(DIRECT means that the hidden state data would not be calculated by other nodes of network as input data states do.)

2. Does the RNN unit mentioned in the original NEAT paper run under the same mechanism as neat-gru? Because the paper doesn't explicitly say how the hidden state is treated in the following states when using NEAT for a decision process problem.

[sakex]

Here the previous input as well as the hidden state are saved in RAM. This means that after calling NN::compute(), the hidden state of the neural net changes and calling NN::compute() again with the same input is very likely to yield a different result.

You can look at the interactive example on https://senges.ch as well as read the code for more details.

[dbsxdbsx]

Thanks, I would dig into the code. By the way, do you think NEAT+GRU is good enough for trading strategy? No need for help from reinforcement learning?

[sakex]

Technically the NEAT algorithm is reinforcement learning. I would say that trading is a very difficult problem with many possible approaches. And you will be competing with very well resourced and insanely talented hedge funds so it is probably not enough in itself, but very worthwhile as part of a larger strategy

[dbsxdbsx]

I have to say, if treaing NEAT as reinforcement learning, it has to mention that NEAT improves network some how less directionally. Anyway, your answer is really good enough to me, thanks