The example throwing

[GeminiSquishGames]

While trying to run the example I get this:

Invalid operands 'Array' and 'float' in operator '+' mutate() @ line 22:

buffer.weights[i] += randf_range(-curiosity_rate, curiosity_rate)

[haloraphi]

I got the same Error when I tried using RLNET. When I added print(buffer.weights[i]) it printed one Array containing 16 Arrays. My network structure was [16,32,32,3]

[braxus32]

Same bug here. Attempted to account for nested arrays but it must be stuck in a loop because the debugger won't show more than the logo when run.

[braxus32]

Just solved the issue, changed the loop in the mutate function to this. You're welcome! Sorry for the formatting.

while i < size: if typeof(buffer.weights[i]) == TYPE_ARRAY: var j := 0 as int while j < buffer.weights[i].size(): if typeof(buffer.weights[i][j]) == TYPE_ARRAY: var k := 0 as int while k < buffer.weights[i][j].size(): buffer.weights[i][j][k] += randf_range(-curiosity_rate, curiosity_rate) k += 1 j += 1 else: buffer.weights[i][j] += randf_range(-curiosity_rate, curiosity_rate) j += 1 i += 1 else: buffer.weights[i] += randf_range(-curiosity_rate, curiosity_rate) i += 1

[ZeroLikviz]

Hi braxus32. I am aware of this problem and fixed it a long time ago, but haven't released an update. The main reason for this is that RLNNET does not actually use RL, it is a simple mutation algorithm. I'll try to recreate the RLNNET class from scratch so that it uses real RL, though I'm not sure if I know enough to make it. I'll also optimise a lot of things. Unfortunately, I don’t have free time to do this now; a lot of exams appeared suddenly. I'll continue working on this repository over the summer holidays. The update will be released by June 20th.

[GeminiSquishGames]

If you are running into trouble with model-based reinforcement learning, you could try implementing Q-Learning to prototype for now if it clicks better and should be more robust than what you have. Academic responsibilities first, but it's good to see you still want to work on it and contribute something to Godot. I've just been using Markov Chains for brains with adaptive weights for scores and state decisions but I'd really like to try your approach and addon in a multi-population life-simulation project at some point.

https://en.wikipedia.org/wiki/Q-learning https://www.baeldung.com/cs/reinforcement-learning-neural-network

[braxus32]

Hey Zero! Happy to hear that there'll be more updates soon. I'm very thankful that you've taken the time to do something like this and share it with the community, it really helped me out with my capstone. I'm looking forward to the rework, but just thought I'd share a quick hotfix for the issue if it might help other people. Good luck with your academics!

[Remi123]

Hi @ZeroLikviz, It seems that the RLNN example is also diverging, even with the modification provided by @braxus32 , as is I'm only able to make it reach 0.0 around 33% of the time. The rest it diverge to 0.9999.

I've just read the comment that around the 20th an update should be released so I'll wait for your update.

[ZeroLikviz]

@GeminiSquishGames, @Remi123, @braxus32, @haloraphi,

The addon has been recently updated. Currently, it includes only a fully rewritten version of NNET. I will be working on PPONNET until June 20th.

@Remi123, this is the reason why I am rewriting everything.