Error in Antmaze environments

ZishunYu / Actor-Critic-Alignment

Implementation of ``Actor-Critic Alignment for Offline-to-Online Reinforcement Learning''

https://proceedings.mlr.press/v202/yu23k.html

MIT License

12 stars 0 forks source link

Error in Antmaze environments #2

Open QinwenLuo opened 6 months ago

QinwenLuo commented 6 months ago

When I run run_aca.py for antmaze, this error will occur: AttributeError: module 'd3rlpy.preprocessing.reward_scalers' has no attribute 'ConstantShiftRewardScaler'

My conda environment satisfy all the requirements listed in requirementstxt, maybe the requirement for d3rlpy is wrong?

ZishunYu commented 5 months ago

Hi Qinwen

Sorry for the late response as I was on vacation last month (due to the new year). Yes, it seems d3rlpy 1.x does not have the ConstantShiftRewardScaler. Could you try StandardRewardScaler with manually setting the mean and std?

scaler = d3rlpy.preprocessing.StandardRewardScaler(mean=[ConstShift], std=1.0)

This should achieve desired reward transformation.