Cycle-Dynamics

This repository contains the official implementation for Cycle-Dynamics introduced in the following paper:

Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency

Qiang Zhang, Tete Xiao, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang

The project page with video is at https://sjtuzq.github.io/cycle_dynamics.html.

Citation

If you find our work useful in your research, please cite:

@article{zhang2020learning,
  title={Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency},
  author={Zhang, Qiang and Xiao, Tete and Efros, Alexei A and Pinto, Lerrel and Wang, Xiaolong},
  journal={arXiv preprint arXiv:2012.09811},
  year={2020}
}

Environment

Python 3.6
PyTorch 1.0 or higher, with NVIDIA CUDA Support
Other required python packages are specified by requirements.txt.

Prerequisites

Install Mujoco and other required packages.
Clone this repository.
Create the log folder './logs/cross_physics', './logs/cross_modality' and './logs/cross_morphology'.

Reproducing Cross-physics Experiment

1. Train the policy network, and collect the source domain data. Or you can download the pretrained policy weight here. Then place it in './logs/cross_physics/', unzip this file and collect the source domian data.

cd cross_physics/base_train_test/td3_solver
python train.py --env 'HalfCheetah-v2'
cd ../../cycle_transfer
python collect_data.py --data_type 'base' --data_id 1

2.Modify the xml file to change the environment, then collect data

cd cross_physics/base_train_test/mujoco_xml
python modify_xml.py --env 'half_cheetah' --phy 'arma3'
cd ../../cycle_transfer
python collect_data.py --data_type 'arma3' --data_id 1

3.Train the model to learn the correspondence

cd cross_physics/cycle_transfer
python forwardexp.py --data_type1 'base' --data_id1 1 --data_type2 'arma3' --data_id2 1

Reproducing Cross-modality Experiment

1. Train the policy network and collect the source domain data. Or you can download the pretrained policy weight here. Then place it in './logs/cross_modality/', unzip this file and collect the source domian data.

cd cross_modality/mujoco_exp/base_train_test/td3_solver
python train.py --env 'HalfCheetah-v2'

2.Collect the unpaired data for both domains.

cd cross_modality/mujoco_exp/cycle_transfer/
python collect_data.py --env 'HalfCheetah-v2' --data_type 'base' --data_id 1

3.Train the model to learn the correspondence.

cd cross_modality/mujoco_exp/cycle_transfer/
python cycleexp.py --data_type1 'base' --data_id1 1 --data_type2 'base' --data_id2 1

Reproducing Cross-morphology Experiment

1. Train the policy network and collect the source domain data. Or you can download the pretrained policy weight here. Then place it in './logs/cross_morphology/', unzip this file and collect the source domian data.

cd cross_morphology/base_train_test/td3_solver
python train.py --env 'HalfCheetah-v2'
cd ../../cycle_transfer
python collect_data.py --data_type 'base' --data_id 1

2.Modify the xml file to change the agent morphology.

cd cross_morphology/base_train_test/mujoco_xml
python modify_xml.py --env 'half_cheetah' --phy '3leg'
cd ../../cycle_transfer
python collect_data.py --data_type '3leg' --data_id 1

3.Train the model to learn the correspondence.

cd cross_morphology/cycle_transfer
python alignexp.py --data_type1 'base' --data_id1 1 --data_type2 '3leg' --data_id2 1

Reproducing Real Xarm Robot Experiment

Download the dataset here.

To train and evaluate the model:

cd realxarm_exp/xarm_pose
python main.py

Toy Experiment

The code for toy experiment is shown in the path './toys', which shows how we explore step by step to get the final framework. This part is not cleaned up but you can still have a reference if you are interested in.

sjtuzq / Cycle_Dynamics

readme