search

liuzuxin / DSRL

🔥 Datasets and env wrappers for offline safe reinforcement learning
https://offline-saferl.org
Apache License 2.0
65 stars 4 forks source link

Unbounded actions in datasets #5

Closed qianlin04 closed 3 months ago

qianlin04 commented 5 months ago

Hi,

I find that action values in some datasets (e.g., BallCircle, CarCircle, DroneCircle) are out of bounds of environments (i.e., [-1, 1]). However, the actions in D4RL are bounded, and when using train_trpol_expert.py to collect new data, we also get bounded actions.

So I want to know if everything is okay. Should I directly clip the actions when I use the datasets?

zgong11 commented 3 months ago

Hi,

I find that action values in some datasets (e.g., BallCircle, CarCircle, DroneCircle) are out of bounds of environments (i.e., [-1, 1]). However, the actions in D4RL are bounded, and when using train_trpol_expert.py to collect new data, we also get bounded actions.

So I want to know if everything is okay. Should I directly clip the actions when I use the datasets?

@liuzuxin @Ja4822 @HenryLHH @yihangyao Same issues here. May I know how do you resolve it? @qianlin04

liuzuxin commented 3 months ago

Hi @zgong11 and @qianlin04 , sorry for the oversight. For some BulletSafetyGym environment, there are indeed such issues. You can either clip the actions in the datasets or just ignore them, because the BulletSafetyGym environment will clip the actions to the correct range [-1, 1], as shown here.

qianlin04 commented 3 months ago

Hi @liuzuxin , thanks for your reply. Actually, I'm a bit confused about the difference between the publicly available datasets and the ones generated by code, but there don't seem to be any unexpected issues when I directly use the clipped actions in my project.