tkoolen
commented
7 years ago Thanks for contacting me about this. I think this package could eventually be good for what you want:
- pure Julia solution with pretty minimal dependencies
- gradients with respect to both parameters (masses, lengths, etc.) and state available (and tested) using e.g. ForwardDiff, enabling policy gradient methods
- reasonably performant at this point
However, it's currently lacking some features that you probably need:
- Contact (looking at the Box2D demos, I'm assuming you'd want this). This is a big missing feature and will take a lot of work to implement. I'm hoping to have some time around the end of the year to do a first pass, but no guarantees.
- https://github.com/tkoolen/RigidBodyDynamics.jl/issues/86 for proper integration of quaternion floating joints (since you're interested in 3D mechanisms). I'd like to add a quaternion-aware velocity Verlet style integrator soonish that should fix this.
- A way to specify input torques in
dynamics!. This is a quick thing that I can add tonight (and honestly should have already).
Things like the cart-pole and pendulum environments you have in Reinforce should be pretty easy to set up after I fix number 3 above, as would any other non-floating mechanism that isn't supposed to contact its environment.
Be sure to check out this example notebook if you haven't already: https://github.com/tkoolen/RigidBodyDynamics.jl/blob/master/examples/Mechanism.ipynb (although it could use some comments...). I should put together an example including 3D visualization as well; I'll try to do that within the next two days. Do keep in mind that I can't work full-time on this package (although I'd like to).
tbreloff
I'm potentially interested in a Julia solution for Deep Reinforcement Learning with robotic simulation. Right now I'm using the Box2D environment for continuous control problems using my packages Reinforce and my wrapper around OpenAIGym.
Your package(s) sound like a possible alternative/solution, especially when considering 3D simulations, but I'm having a hard time wrapping my head around the easiest way to make a simple reinforcement learning enviroment using RigidBodyDynamics. Does this sound like a good use for your software? Can you help me get started (or better yet, do you have interest collaborating on building environments?)