search

AryamanReddi99 / quantal-adversarial-rl

This repository implements the code for the paper 'Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula'.
1 stars 0 forks source link

Cannot reproduce the performance #1

Open buka7yo opened 2 months ago

buka7yo commented 2 months ago

Thank you for releasing the code! I really appreciate your work on this project. However, I encountered a few issues when trying to run it:

  1. The files in the "teacher" folder require importing GaussianTorchDistribution from distributions.torch_distributions. Is the distributions.torch_distributions module the one located in the mushroom_rl_extensions folder?
  2. I ran QARL on the Cartpole - Swingup Sparse task, but the return for QARL is only 104, and the return against the adversary is 0. My command was: python3 -m main --algorithm qarl --domain_name cartpole_two_players --task_name swingup_sparse_vs_adversary --use_cuda True.

Thank you in advance for your assistance!

AryamanReddi99 commented 1 month ago

Hi, thanks for your interest in QARL!

  1. Yes, that's the correct module - use the GaussianTorchDistribution provided with mushroom_rl_extensions in the source code.
  2. That's interesting - could you please share the rest of the hyperparameters you used? Keep in mind that Cartpole - Swingup Sparse is a very sensitive environment so it's not unusual to see 0 performance against an adversary for any training algorithm, curriculum or otherwise!