search

Lyceum / MuJoCo.jl

MIT License
45 stars 0 forks source link

wrapper for derivative.cpp #17

Open thowell opened 3 years ago

thowell commented 3 years ago

Awesome work!! Would love to use these packages for model-based control, is there functionality for derivatives implemented anywhere in the Lyceum ecosystem?

klowrey commented 3 years ago

I'm actually working on it now to jive well with Lyceum, but it's a little challenging to still retain efficiency with auto-diff. Shouldn't be too long now, however.

copying the code from the derivative.cpp example to just use MuJoCo.jl is feasible as well, but doesn't benefit from the extended ecosystem.

thowell commented 3 years ago

Any progress on this? If you can give me some pointers, I might be able to implement this functionality.

klowrey commented 3 years ago

The tricky part is the interaction between an AbstractEnv and MJSim, since sometimes one or the other needs the differentiation. To handle it I'm using ChainRulesCore.jl composites to manually handle what fields we're differentiating for.

If you just wanted a port of derivative.cpp from the mujoco sample code, I have that around here as well but that's if you're working more with MuJoCo rather than the Lyceum interface.

klowrey commented 3 years ago

Any progress on this? If you can give me some pointers, I might be able to implement this functionality.

What use case did you need derivatives for? You mention mode based control so that could be as simple as derivatives for mj_forward / mj_inverse, or if you wanted generality that would be like differentiating through the Lyceum Api.

It would be nice to know if some other work flow so I'm not missing use cases as I'm trying to get autodiff to play well.

thowell commented 3 years ago

just the basic derivatives, this would be for the mj_forward and mj_inverse. we want to do model-predictive control (e.g., using iterative LQR)

klowrey commented 3 years ago

https://gist.github.com/klowrey/b9ba2e7194e4aa30bd12ee9cc393bc72 here is a port of the derivative.cpp code to julia. it includes the analytical way of checking between inverse and forward and should be the same as derivative.cpp.

as a note, sometimes the Julia threading overhead is larger than pthreads/c++ so depending on your application you may want to run it single threaded and parallelize with other means.

If you're using ILQG from https://github.com/baggepinnen/DifferentialDynamicProgramming.jl I'd love to know how it turns out. I never had much luck with iLQG for complex models; MPPI often worked better and the reward function was easier to debug, all without needing derivatives.

I was converting lyceum to be automatically differentiable in order to use it through Neural ODE frameworks, which is an adjecent idea to trajectory optimization.

mfogelson commented 3 years ago

It seems like some functions are missing from MuJoCo.MJCore, like mju_quatIntegrate. so I cant run get_acceljac

klowrey commented 3 years ago

I made a branch of this repo titled quat for those functions, but have not merged it into master to avoid potential breakings.