Add a few sentences to the docstring of TDAgent.incremental_update() to clarify what's going, e.g. with respect to which state is St and which state is St+1. I suspect that we'll be doing ourselves a favour by doing it sooner rather than later.
@RasmusBrostroem do you still have our diagram on your whiteboard? If so, could you please attach a picture of it in a comment? If we write a short report on the TD-method, it might be good to include (a revised version of) it.
Add a few sentences to the docstring of
TDAgent.incremental_update()
to clarify what's going, e.g. with respect to which state is St and which state is St+1. I suspect that we'll be doing ourselves a favour by doing it sooner rather than later.@RasmusBrostroem do you still have our diagram on your whiteboard? If so, could you please attach a picture of it in a comment? If we write a short report on the TD-method, it might be good to include (a revised version of) it.