Denys88 / rl_games

RL implementations
MIT License
848 stars 142 forks source link

Clamped Actions Between [-1,1] #299

Closed rogerpm3 closed 1 month ago

rogerpm3 commented 1 month ago

I have an issue exporting a trained model as with the following configuration the actions values are clamped between [-1 , 1] (I don't understand why but also happens with all the other examples) I can work with that, however when exporting I get values with no limits so it's no use. How can I solve this.

params:
  seed: ${...seed}
  algo:
    name: a2c_continuous

  model:
    name: continuous_a2c_logstd

  network:
    name: actor_critic
    separate: False
    space:
      continuous:
        mu_activation: None
        sigma_activation: None
        mu_init:
          name: default
        sigma_init:
          name: const_initializer
          val: 0
        fixed_sigma: True
    mlp:
      units: [512, 256, 128]
      activation: elu
      d2rl: False
      initializer:
        name: default
      regularizer:
        name: None

  load_checkpoint: ${if:${...checkpoint},True,False} # flag which sets whether to load the checkpoint
  load_path: ${...checkpoint} # path to the checkpoint to load

  config:
    name: ${resolve_default:Tiago,${....experiment}}
    full_experiment_name: ${.name}
    env_name: rlgpu
    multi_gpu: ${....multi_gpu}
    ppo: True
    mixed_precision: False
    normalize_input: True
    normalize_value: True
    num_actors: ${....task.env.numEnvs}
    reward_shaper:
      scale_value: 0.01
    normalize_advantage: True
    gamma: 0.99
    tau: 0.95
    learning_rate: 3e-4
    lr_schedule: adaptive
    kl_threshold: 0.008
    score_to_win: 100000000
    max_epochs: ${resolve_default:30000,${....max_iterations}}
    save_best_after: 200
    save_frequency: 1000
    print_stats: True
    grad_norm: 1.0
    entropy_coef: 0.0
    truncate_grads: True
    e_clip: 0.2
    horizon_length: 32
    minibatch_size: 512
    mini_epochs: 8
    critic_coef: 4
    clip_value: True
    seq_len: 4
    bounds_loss_coef: 0.0001