[Feature Request] Refactor mujoco_env.h with dm composer callback API - Githubissues

sail-sg / envpool

C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

https://envpool.readthedocs.io

Apache License 2.0

1.1k stars 100 forks source link

[Feature Request] Refactor mujoco_env.h with dm composer callback API #103

Open Trinkle23897 opened 2 years ago

Trinkle23897 commented 2 years ago

Motivation

https://github.com/sail-sg/envpool/issues/59#issuecomment-1110289093

We are going to refactor the MujocoEnv class with deepmind composer API. It is more general with various kinds of robotics research.

Resources

https://arxiv.org/pdf/2006.12983.pdf page 16
composer task: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/task.py
composer environment: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/environment.py

Benjamin-eecs commented 2 years ago

Basically class task serve as hooks in MujocoEnv with following methods:

Class task

[x] initialize_episode_mjcf
[x] after_compile
[x] initialize_episode
[x] before_step
[x] before_substep
[x] after_substep
[x] after_step

DeepMind Control Suite, 14 envs

[x] Acrobot (@Benjamin-eecs, #129)
[x] Ball in cup (@Benjamin-eecs, #124)
[x] Cart-pole (@Benjamin-eecs, #137)
[x] Cheetah (@yufansong, #110)
[x] Finger (@yufansong, #118)
[x] Fish (@Benjamin-eecs, #134)
[x] Hopper (@Trinkle23897, #107)
[x] Humanoid (@Benjamin-eecs, #132)
[x] Manipulator (@yufansong, #128)
[x] Pendulum (@yufansong, #122)
[x] Point-mass (@yufansong, #123)
[x] Reacher (@yufansong, #115)
[x] Swimmer (@Benjamin-eecs, #140)
[x] Walker (@yufansong, #114)

Extra envs in `dm_control.suite`

[ ] Dog
[x] Humanoid_CMU (@Benjamin-eecs, #145)
[ ] Lqr (@Benjamin-eecs )
[ ] Quadruped
[ ] Stacker

`dm_control.locomotion`

[ ] Navigating a corridor with obstacles
[ ] Maze navigation and foraging
[ ] Multi-Agent Soccer

Misc

[x] Apply name2id (@Trinkle23897, #125)
[ ] Class variation
[ ] Class observables
[ ] dm_control.manipulation

Resources

control.Environment: https://github.com/deepmind/dm_control/blob/main/dm_control/rl/control.py
control.Task: https://github.com/deepmind/dm_control/blob/main/dm_control/suite/base.py https://github.com/deepmind/dm_control/blob/main/dm_control/rl/control.py
composer.Environment: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/environment.py
composer.Task: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/task.py

kevinzakka commented 2 years ago

I'll offer feedback as soon as I'm free :) Thanks for initiating!