Automatic differentiation (Autodiff) Feature Roadmap

taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.

https://taichi-lang.org

Apache License 2.0

25.08k stars 2.26k forks source link

Automatic differentiation (Autodiff) Feature Roadmap #5050

Open erizmr opened 2 years ago

erizmr commented 2 years ago

autodiff_roadmap_svg

I would like to propose an initial roadmap of the Automatic differentiation (Autodiff) feature from my perspective. Autodiff is a useful tool for differentiable programming, which support various applications in scientific computing and artificial intelligence.

Currently, Taichi supports reverse mode autodiff. The ongoing efforts include:

developing forward mode autodiff.
exploring computing second order derivative

eatcosmos commented 2 years ago

Any recommendations for books for autodiff, haven't touched this point of knowledge

erizmr commented 2 years ago

Any recommendations for books for autodiff, haven't touched this point of knowledge

Andreas Griewank and Andrea Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation, volume 105. Siam, 2008.