Tile Coding scaling issue

ShangtongZhang / reinforcement-learning-an-introduction

Python Implementation of Reinforcement Learning: An Introduction

MIT License

13.45k stars 4.81k forks source link

Tile Coding scaling issue #116

Closed MJeremy2017 closed 5 years ago

MJeremy2017 commented 5 years ago

In chapter 10, why the state is scaled in self.positionScale = self.numOfTilings / (POSITION_BOUND[1] - POSITION_BOUND[0])? It does not seem like to scale into a 0, 1 range, may I know why scale in this way and is it all usage of the function has to scale in this way?

ShangtongZhang commented 5 years ago

I remember this scaling comes from the description in the book. Not sure if it is still there in the latest version. Typically I would scale it to [0, 1].

roachsinai commented 5 years ago

https://github.com/ShangtongZhang/reinforcement-learning-an-introduction/blob/4d8928bb7538d81b818267c983f3fd004ffc9068/chapter10/mountain_car.py#L134

but if you want scole it to [0, 1], I think the parameter here should be [self.position_scale * position - POSITION_MIN, self.velocity_scale * velocity - VELOCITY_MIN],