search

dojo-sim / Dojo.jl

A differentiable physics engine for robotics
MIT License
309 stars 27 forks source link

Allow using Folds to parallelize `ARS` #7

Closed rejuvyesh closed 2 years ago

rejuvyesh commented 2 years ago

I tried to keep the changes minimal. Folds backend is not default although technically it does support DistributedEx executor too, so would be easy to simplify train to just use Folds for singlethreaded, multithreaded and distributed execution.

Note that with Julia v1.8 Threads.@threads default behavior will change. Moreover, AFAIU, using Threads.threadid() is not recommended as it's not safe (or will not be safe from v1.8). See https://discourse.julialang.org/t/behavior-of-threads-threads-for-loop/76042/28, for more details.

5 is not fixed by this.

Also updated MeshCat and StaticArrays required versions. Let me know if you don't want that.

rejuvyesh commented 2 years ago

Some timing details with 6 threads:

# Threads: current

episode 1 reward_evaluation -14.46746066226647. Took 107.058050829 seconds
episode 2 reward_evaluation -15.771144785726458. Took 69.066143253 seconds
episode 3 reward_evaluation -16.028737198815033. Took 69.246535135 seconds
episode 4 reward_evaluation -10.649176633356072. Took 67.825174628 seconds
episode 5 reward_evaluation -17.217895823210295. Took 69.035207496 seconds
episode 6 reward_evaluation -7.103001627654827. Took 67.607978192 seconds
episode 7 reward_evaluation -9.207607080068437. Took 69.197771062 seconds
episode 8 reward_evaluation -3.5272094497191944. Took 63.919988394 seconds

# Folds: default ThreadedEx()

episode 1 reward_evaluation -16.417007623298435. Took 162.40875345 seconds
episode 2 reward_evaluation -16.232712897839175. Took 98.691055335 seconds
episode 3 reward_evaluation -8.957962721474344. Took 100.598343974 seconds
episode 4 reward_evaluation -10.758372823739798. Took 103.95178815 seconds

# Folds: ThreadedEx(;basesize=1)

episode 1 reward_evaluation -17.54907748855047. Took 80.583351401 seconds
episode 2 reward_evaluation -14.295564082426687. Took 77.502262604 seconds
episode 3 reward_evaluation -3.550502749042572. Took 78.39897174 seconds
episode 4 reward_evaluation -7.492981769568877. Took 84.359059808 seconds
rejuvyesh commented 2 years ago

Failing tests should likely be fixed by #8.