ShangtongZhang / DeepRL

Modularized Implementation of Deep RL Algorithms in PyTorch
MIT License
3.18k stars 682 forks source link

replay buffer: why next state start index is n_steps after state start index? #110

Closed TaciturnMute closed 1 year ago

TaciturnMute commented 1 year ago
    this is the code:

    s_start = index - self.history_length + 1
    s_end = index
    if s_start < 0:
        raise RuntimeError('Invalid index')
    next_s_start = s_start + self.n_step      ###################
    next_s_end = s_end + self.n_step
    if s_end < self.pos and next_s_end >= self.pos:
        raise RuntimeError('Invalid index')