search

brohrer / robot-brain-project

a general purpose learning agent
205 stars 46 forks source link

Implement multi-step lookahead #6

Closed brohrer closed 9 years ago

brohrer commented 9 years ago

In the goal-selection mechanism, choose whether to select a goal based on a parameter value. The parameter represents the urgency of acting.

The grid_1D_ms might a a good first world to test this in.

brohrer commented 9 years ago

I removed this issue from the 0.6.0 release

BECCA's enhancement strategy is very pragmatic: only add features when they can be justified by performance increases in a world. Another way to say it is that if I want to add feature X, I first need to devise a world in which that feature will significantly improve performance.

In order to justify multi-step lookahead, I need to create a complex world with multiple reward timescales that requires multi-step actions to achieve them. I'll include this in a later release.

brohrer commented 9 years ago

grid_1D_ms can cheat by relying on reward traces. To demonstrate multi-step lookahead well, the goal state needs to be one that is only occasionally rewarded.

brohrer commented 9 years ago

I've embarked on a deep reworking of BECCA. It will indeed implement multi-step lookahead, but in a different way than is currently feasible. I'm going to close this task and re-integrate the work into the new roadmap.