Making this work again

[alexswerner]

Hi,

amazing work here. But the software has moved on and I would like to make it work again. So far I have:

• Fixed the code to work with the new pyelastica API
• Updated from stable_baselines to stable_baselines3
• Updated from gym to gymnasium v26 (also required by stable_baselines3)
• Note that stable_baselines3 is based on pytorch
• Parallelized the training for e.g. PPO

Right now I can execute the training for PPO in Case1 which is at least something. This is by directly calling:

python3 logging_bio_args.py --total_timesteps=10000000.0 --SEED=0 --timesteps_per_batch=32000 --algo=PPO

I pushed the code here: https://github.com/alexswerner/Elastica-RL-control

However, the output of the post processing script does not show a moving robot. :-(

So there is still some work to do.

[alexswerner]

The PPO test case seems to work again after some debugging. Key findings:

• Passing an empty self.spline_points_func_array_normal_dir = [] to the constructor of MuscleTorquesWithVaryingBetaSplines does not work as intended.
• Switching to the dampen API using AnalyticalLinearDamper needs apparently a different nu value
• Parallelizing PPO does not yield the nice convergence without further modification of the parameters

One option question for precisely reproducing the test case is how to transfer the nu specified in the old Cosserat rod constructor to the new dampen API.

[armantekinalp]

Hey,

Thank you for your interest.

For the current PyElastica damping implementation, you can scale the nu as nu_new = nu_old / density / base_area