FLAIROx / Kinetix

Reinforcement learning on general 2D physics environments in JAX
https://kinetix-env.github.io/
MIT License
93 stars 0 forks source link

Erroneously static linkages in large environments #4

Open kvablack opened 2 hours ago

kvablack commented 2 hours ago

Hi,

Big fan of this project! I'm trying to train an RL agent on a bunch of large environments at once, and I'm seeing an issue where some linkages are static/immobile when they shouldn't be. Here are HalfCheetah and Pinball:

https://github.com/user-attachments/assets/b804f16b-669d-44c5-ad41-8e1a77e3898d

https://github.com/user-attachments/assets/97c40b7f-fbde-4b17-a4d9-9b2a5140e1fd

The problem exists on hard_pinball, cartpole, mjc_walker, Unicycle, mjc_swimmer, mjc_half_cheetah, mjc_hopper, and grasp_easy. If I download the JSON files and load them into Kinetix.js, they work normally.

Here's my config file to reproduce, used with experiments/ppo.py:

defaults:
  - env_size: l
  - learning: ppo-base
  - misc: misc
  - eval: eval_auto
  - model: model-base
  - eval_env_size: l
  - _self_

misc:
  video_frequency: 30
  save_policy: false

env:
  env_name: "Kinetix-Symbolic-MultiDiscrete-v1"
  dense_reward_scale: 1.0
  frame_skip: 2

train_levels:
  train_level_mode: list
  train_levels_list:
    - "l/grasp_easy.json"
    - "l/car_swing_around.json"
    - "l/h4_cartpole.json"
    - "l/h15_swing_up.json"
    - "l/hard_lunar_lander.json"
    - "l/mjc_half_cheetah.json"
    - "l/mjc_hopper.json"
    - "l/mjc_swimmer.json"
    - "l/mjc_walker.json"
    - "l/chain_lander.json"
    - "l/h17_unicycle.json"
    - "l/h16_thruster_goal.json"
    - "l/hard_pinball.json"
    - "l/h5_flappy_bird.json"
    - "l/rail.json"
    - "l/chain_thrust.json"

eval:
  eval_freq: 40
learning:
  num_steps: 256
  filter_levels: false
seed: 0
kvablack commented 2 hours ago

Update: I'm guessing a thruster is also broken in thruster_goal, since this environment should be fairly trivial to solve, but it hasn't been yet:

https://github.com/user-attachments/assets/e8f70b2f-ac86-4897-b5b2-b1e9e60db93b