search

erwincoumans / tiny-differentiable-simulator

Tiny Differentiable Simulator is a header-only C++ and CUDA physics library for reinforcement learning and robotics with zero dependencies.
Apache License 2.0
1.23k stars 130 forks source link

add optimized CPU training examples for Ant and Laikago, using OpenMP #183

Closed erwincoumans closed 3 years ago

erwincoumans commented 3 years ago 
examples/ars/ars_train_policy_omp_ant.cpp
refactor examples/ars/ars_vectorized_environment.h to be used for either CUDA or CPU/OpenMP.
add examples/environments/ant_environment2.h and examples/environments/laikago_environment2.h that use codegen optimizations
opengl_urdf_visualizer: add convert_visuals method that creates a single instance of each, for ease of use (if you don't need multiple instances)

allow to use clang with OpenMP on MacOS (tested on Mac M1 Pro (arm)) brew update brew install llvm libomp cmake -DCMAKE_C_COMPILER="/opt/homebrew/opt/llvm/bin/clang" -DCMAKE_CXX_COMPILER="/opt/homebrew/opt/llvm/bin/clang++" -DUSE_APPLE_HOMEBREW_CLANG=ON ..

erwincoumans commented 3 years ago

On a MacBook M1 Pro (14") with 8 Cores (6 performance, 2 efficiency) the training of a custom Ant runs at almost 100k steps per second, using OpenMP and generated C code using CppAdCodegen. Compile using above home-brew clang and then run examples/ars/ars_train_policy_omp_ant

Time    Iteration       AverageReward   MaxRewardRollout        MinRewardRollout        timesteps
23.93    10 818.099707  1074.45    42.9172    1932001
51.526    20 275.611223  679.283    -403.851    4333386
79.205    30 1024.878382  1303.88    106.159    6824583
106.478    40 1276.533419  1570.6    -249.706    9472280
133.704    50 1756.318418  1846.07    1428.89    12146026
162.837    60 1778.257093  1884.86    1524.56    14825677