search

Toni-SM / skrl

Modular reinforcement learning library (on PyTorch and JAX) with support for NVIDIA Isaac Gym, Omniverse Isaac Gym and Isaac Lab
https://skrl.readthedocs.io/
MIT License
443 stars 43 forks source link

Fix TD3 DDPG Implementation: Move Sampling Inside Gradient Step Loop #144

Closed alessandroassirelli98 closed 2 months ago

alessandroassirelli98 commented 3 months ago

This pull request addresses a discrepancy between the original TD3 and DDPG paper's algorithm and the current implementation in the repository. Specifically, the original implementation performs the sampling step outside of the gradient step loop, which diverges from the methodology outlined in the paper. We have corrected this by moving the sampling process inside the gradient step loop, aligning the implementation more closely with the intended algorithmic procedure described in the original paper and SpinningUp description.

Toni-SM commented 2 months ago

Hi @alessandroassirelli98

Could you please, update the PR target branch to the develop branch. The main branch is updated from the develop branch only when a new release is made :)

alessandroassirelli98 commented 2 months ago

Ok, I have created another Pull Request for branch develop

Toni-SM commented 2 months ago

Nice... This PR will be closed then in favor of https://github.com/Toni-SM/skrl/pull/147