search

vojtamolda / reinforcement-learning-an-introduction

Solutions to exercises in Reinforcement Learning: An Introduction (2nd Edition).
340 stars 74 forks source link

[Exercise 04.07] possible issue while calculating the transfer_cost if not all the cars indicated by the action can be transferred #23

Open duxtinto opened 1 year ago

duxtinto commented 1 year ago

Hi @vojtamolda !

While debugging the code for https://github.com/vojtamolda/reinforcement-learning-an-introduction/blob/main/chapter04/exercise04-07.ipynb, I got into this scenario.

On the transitions method of the JacksCarRental class: image

(1) the value of transfer was -5 (2) the transfer_cost was 5*2=10 (3) the transferred value was -3 (as self.max_cars - state[0] was 3)

it looks weird to me that the transfer_cost is for the 5 cars, but in fact we only moved 3.

shouldn't the trasnfer_cost be calculated only for the transferred cars?

image

Thank you very much for your help.

Best regards,

David.

vojtamolda commented 1 year ago

Thanks for opening the issue. I don't have time to work on this right now, but I'll look into it eventually.