DiffMimic:
Efficient Motion Mimicking with Differentiable Physics

Jiawei Ren^* Cunjun Yu^* Siwei Chen Xiao Ma Liang Pan Ziwei Liu^†

S-Lab, Nanyang Technological University National University of Singapore
^*equal contribution
^†corresponding author

ICLR 2023

---

[Project Page] • [Paper] • [Demo] • [Video]

About

We implement DiffMimic with Brax:

Brax is a fast and fully differentiable physics engine used for research and development of robotics, human perception, materials science, reinforcement learning, and other simulation-heavy applications.

An environment mimic_env is implemented for training and benchmarking. mimic_env now includes the following characters:

HUMANOID: AMP-formatted humanoid, used for acrobatics skills.
SMPL: SMPL-formatted humanoid, used for mocap data.
SWORDSHIELD: ASE-formatted humanoid, used for REALLUSION sword-shield motion.

More characters are on the way.

Installation

conda create -n diffmimic python==3.9 -y
conda activate diffmimic

pip install --upgrade pip
pip install --upgrade "jax[cuda]==0.4.2" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
pip install -r requirements.txt

Get Started

python mimic.py --config configs/AMP/backflip.yaml

Visualize

streamlit run visualize.py

Citation

If you find our work useful for your research, please consider citing the paper:

@inproceedings{ren2023diffmimic,
  author    = {Ren, Jiawei and Yu, Cunjun and Chen, Siwei and Ma, Xiao and Pan, Liang and Liu, Ziwei},
  title     = {DiffMimic: Efficient Motion Mimicking with Differentiable Physics},
  journal   = {ICLR},
  year      = {2023},
}

Acknowledgment

Differentiable physics simulation is done by Brax.
Early version of the code is heavily based on Imitation Learning via Differentiable Physics (ILD).
Motion files are borrowed from DeepMimic, ASE, AMASS, and AIST++.
Characters are borrowed from DeepMimic and ASE.
The work is inspired by valuable insights from SuperTrack and Spacetime Bound.

jiawei-ren / diffmimic

readme