jonas-eschmann
commented
2 weeks ago Hi @smartengm
- Do you mean a differentiable model for an MPC? MPCs usually don't have the cocept of an (amortized) policy
- Like mentioned in https://github.com/rl-tools/rl-tools/issues/6#issuecomment-2152531312 it should be relatively straightforward to move the code to the ESP32 you are mentioning. The most important point is that it needs enough memory ~1.3MB iirc for the SAC Pendulum example. Note that, by just looking at the clock speed, the ESP32 is probably around 3x slower than the Teensy
- The Teensy is solid for training, for inference any (even the smallest ones like the STM32F405 in the Crazyflie) are more than enough
Dear Sir,
First of all, I should appreciate your team for creating RLtools.
Here are my questions: 1- Can I use RLtools for implementing Differentiable MPC policy for deep reinforcement learning on microcontrollers?
2-It is very important for me to have training on microcontrollers periodically, when I use RLtools for deep reinforcement learning. On your website, it seems that you used Teensy 4.1 for training . On the other hand , you used esp32 for inferencing . Can I use new esp32-s3 series(which have reasonable RAM and Flash memory) for training, just like your example for Teensy 4.1?
3-In your opinion, who has worked a lot in the realm of deep reinforcement learning, which industrial boards(or microcontrollers) are suitable for implementing deep reinforcement learning algorithms? What are your suggestions?
Your consideration is highly appreciated.
Best Regards, smartengm.