Why different discounted rewards for different genomes?

CodeReclaimers / neat-python

Python implementation of the NEAT neuroevolution algorithm

BSD 3-Clause "New" or "Revised" License

1.42k stars 492 forks source link

Why different discounted rewards for different genomes? #117

Open mdalvi opened 6 years ago

mdalvi commented 6 years ago

Hello CodeReclaimers,

I was taking a look at evolve.py in the openai-lander example and noticed the use of different discounts for each genome.

My question is, what is the significance of using different discounts for each genome? Why not use some arbitrary constants like

m = 50
discount = 0.9
discount_function = [discount ** (m - i) for i in range(m + 1)]

Won't this favor some genomes while penalizing others lifelong during training?

KarlXing commented 5 years ago

I think the discount attribute is also considered as a parameter to evolve. In the LanderGenome class, you can see that the discount is also involved in mutate, crossover, configure new and distance calculation.