Dual clipping parameter

opendilab / PPOxFamily

PPO x Family DRL Tutorial Course（决策智能入门级公开课：8节课帮你盘清算法理论，理顺代码逻辑，玩转决策AI应用实践）

https://opendilab.github.io/PPOxFamily/

Apache License 2.0

1.85k stars 168 forks source link

Dual clipping parameter #95

Closed jviquerat closed 2 months ago

jviquerat commented 2 months ago

Hi,

I might be wrong, but I believe the dual clipping parameter is intended to be > 1:

Screenshot 2024-04-26 at 18 27 03

https://github.com/opendilab/PPOxFamily/blob/79761dc67ac812020c2f8908b2612a8f9a893014/chapter7_tricks/dual_clip.py#L59

PaParaZz1 commented 2 months ago

This is indeed our problem, here the dual_clip value should be greater than 1.0. We are sorry about the problem when migrating code from DI-engine (link). We will fix this problem soon.

PaParaZz1 commented 2 months ago

We have fixed this problem in #96.