search

nicklashansen / dmcontrol-generalization-benchmark

DMControl Generalization Benchmark
MIT License
166 stars 39 forks source link

How to implement data-mixing only mentioned in the SVEA paper? #6

Closed ghost closed 2 years ago

ghost commented 2 years ago

Congratulation to be accepted by NIPS 2021!! I wanna ask how to implement data-mixing only mentioned in the SVEA paper?

nicklashansen commented 2 years ago

Hi, are you referring to our "data-mixing only" ablation? For this ablation, we simply concatenate augmented and unaugmented observations along the batch dimension (for both s_t and s_t+1) after sampling. The other ablation (alpha=0, beta=1) is equivalent to only using augmented copies of s_t and only unaugmented copies for the Q-target.

ghost commented 2 years ago

Thanks! Do you mean that the s_t+1 is also augmented for "data-mixing only" ablation? And what's alpha and beta for "data-mixing only" ablation?

nicklashansen commented 2 years ago

Correct. Our two ablations ablate each of the two components of SVEA (data-mixing and unaugmented Q-targets). We implemented the "data-mixing only" baseline using the same hyperparameters as for our proposed method (i.e., alpha=0.5, beta=0.5). Translated into code it would be something like this: obs = utils.cat(obs, augmentations.random_conv(obs.clone())); next_obs = utils.cat(next_obs, augmentations.random_conv(next_obs.clone())) following the code style in https://github.com/nicklashansen/dmcontrol-generalization-benchmark/blob/main/src/algorithms/svea.py.

ghost commented 2 years ago

Thanks a lot !

nicklashansen commented 2 years ago

Happy to help! I am closing this issue; feel free to open a new issue if you have further questions