Does it guarantee to converge?

[w3ntao]

I deploy your strategy to my program:

//update when death:

Q[lastS][lastA] = (1 - lrAlpha) * Q[lastS][lastA] + lrAlpha * deadReward;

for all (S, A) in reversed(expHistory):
    Q[S][A] = (1 - lrAlpha) * Q[S][A] + lrAlpha * (aliveReward + gamma * max(Q[S_]));
    S_ = S;

My bird hit highest score at about 500 round, then scores keep going down. At about 1000 round, it never make through more than 1 pipe and keep hitting ceiling or upper pipe.

Have you encounter such problem?

[chncyhn]

Hello @WentaoZero, I don't remember having this problem.

Does this happen consistently for all runs? Also have you tried using different hyperparameters (learning rate etc.) and discretization levels?

[w3ntao]

Hi @chncyhn , good to have you replied. That was a week ago, now I already have it fixed. This is my repo: https://github.com/WentaoZero/q-bird And your work is cited 😄

[chncyhn]

@WentaoZero That's great to hear! Very nice project. 👍