TODO hopper robot

[mfocchi]

• [x] (Michele Andrea) jump optimization with crocoddyl for comparison
• [x] (Riccardo) RL with torques, give low reward with target distance even if TD is not achieved to encourage moving toward target
• [x] (Ricccardo) Eval extrapolation quality with Max Number of training episodes. Create Feasible region: generate random inference set of N points with the same seed, inside thetraining region, and use that to compute reward that is associated with a color code to each target,
• [x] (Riccardo) Plot different feasible regionr (expect assymmetric) with different number of episodes
• [ ] (Michele Riccardo) Create pointcloud map from 3d rough terrain (eval targetZ (X,Y))
• [x] (Riccardo) Repeat training with torque limits 2 Future:
• [ ] Robustness: add uncertainty on link masses during training phase (expect to get more conservative trajectories)
• [ ] Robustness: start jump to test terrain compliance, do learning with different terrain compliances
• [ ] Simulate multiple robots with coppelia sim that update the same target network

[mfocchi]

1) no issue of self collision as in butter paper https://arxiv.org/pdf/2109.11978.pdf 2) robust learning with representative parameter obtained with heuristic template motion. uncertainties of the physical parameters, e.g., dynamics randomization, can be viewed as external forces applied to the robot(Sim-to-real: Learning agile locomotion for quadruped robots) 3) computational time is spent on contact detection and handling. Reduc-ing the number of potential contact pairs increases simulation throughput.We optimize the modelof the robot to keep only the necessary collision bodies: feet, shanks, knees, and the base. 4) To bridge the reality gap, a series of measures such as increasing the noise level, adding latency and domain randomization are implemented. We assume Gaussian distribution to be present on the real robot for all quantities in the observation space. 5) learn also the controller parameters

[mfocchi]

• [x] try with smaller neurons (64) to see the training gets slower (find smallest network size)
• [ ] evaluate different simulators to run multirobot that mutually update same target policy . These are new API of openAI Gymhttps://github.com/Farama-Foundation/Gymnasium
• [ ] extend to quadruped
• [ ] do training for robustness in simulation and do domain improvement in experiments

[mfocchi]

• [x] add tracking of position com_lo
• [x] do all tests with the final parameytric setup (layers/neurons/number of episodes / compare with / without feasibility check reward)
• [x] torques training (scaling) (does not work)
• [x] optimization
• [x] add plot of error for different layers / ideal landing cost (jump feasibility check)
• [x] put relative error %
• [x] add translucid convex hull RPE 50 VS RPE 10 for 50k episodes
• [x] separate front graph back graph
• [x] do bigger test zone
• [x] redo video with few balls

[mfocchi]

New IIT exchange 1) extend to quadruped 2) swing optimization 3) learn to jump while not static (e.g. running) https://github.com/Improbable-AI/rapid-locomotion-rl : try Isaac gym 4) test with deformable trampuline

• [ ] fix jacobian github.com/NVIDIA-Omniverse/orbit/ issues/267
• [x] change step function to be updated when we want
• [x] ffwd kinematics
• [ ] gravity compesnation lower gains

[mfocchi]

do jump down from pallet

[mfocchi]

implement controller with lift off legs

[mfocchi]

Comparison: 1) learning different policies with / without rearing 2) different low levels PD / MPC 3) with without delta tau [ ] learn a single jump forward (only linear), do grid search on reward weights (target, tochdown)