search

pybrain / pybrain

BSD 3-Clause "New" or "Revised" License
2.85k stars 789 forks source link

egreedy: no consideration of exploration rewards #27

Closed schmalerc closed 14 years ago

schmalerc commented 14 years ago

Hi. I'm not sure if this is considered an issue and therefore in the right place here. The rewards received during random exploration with egreey agent do not seem to have any effect on the action values. Also, exploration is not listed in the history of the agent.

Adding self.lastaction = action in getAction() of egreedy.py seems to solve this. I think the results of the maze example in Q-learning.py make more sense this way.

http://github.com/schmalerc/pybrain/blob/90204cab0ec47c3abb8ec3c5a3df236b9d1863a5/pybrain/rl/agents/egreedy.py

rueckstiess commented 14 years ago

Thanks, we're currently working on a complete redesign of the RL framework with exploration as a seperate module. The issue will be solved there, because the agent will store whatever explorative action came out of the explorer module.