search

vojtamolda / reinforcement-learning-an-introduction

Solutions to exercises in Reinforcement Learning: An Introduction (2nd Edition).
340 stars 74 forks source link

Error to Exercise 2.4 #10

Closed earthwuyang closed 2 years ago

earthwuyang commented 2 years ago

Hello @vojtamolda , when I calculate the formula in Exercise 2.4 I found it should be j=i+1. But I'm not sure if I'm right. Could you have a look when you have free time? Thanks. 微信图片_20220213153520

earthwuyang commented 2 years ago

I found another reference the same with mine. But it's in Chinese. https://blog.csdn.net/zte10096334/article/details/83446191

vojtamolda commented 2 years ago

Hi again!

Yeah. I checked it and I think you're right here as well. The summation should start from j = i + 1.

Also, there's a little bit of an "unwritten convention" in there. When i = n and the blue summation bound starts from j = n+1 and terminates with n, the product should be equal to 1. This way it correctly recovers the term $\alpha_n R_n$ that you wrote separately. I tried to include it in the blue summation but it isn't very elegant...

The correct solution looks like this, I think:

Correction