search

keithalewis / epsilon

1 stars 4 forks source link

Isnt this just forward mode automatic differentiation implemented with operator overloading on taylor series. #8

Open MarisaKirisame opened 4 years ago

keithalewis commented 4 years ago

Hi @MarisaKirisame . Thanks for taking an interest. I'm not sure what you mean by 'operator overloading on taylor series'. Can you point me at code I can run to help me understand that?

MarisaKirisame commented 4 years ago

I assume you understand forward mode automatic differentiation via operator overloading, aka dual number. Dual Number is just taylor series truncated at first order epsilon. You build a library that does forward mode automatic differentiation, at arbitrary truncation (or carrying infinitely). This had been done before - see the paper "beautiful differentiation", for example. Or, a better way to say this is, this is how most higher order automatic differentiation algorithm work.

keithalewis commented 4 years ago

I am familiar with Conal Elliot's work and find it quite impressive, not just in AD. It is fun to write math papers but it is more rewarding to implement the math in software. The theory has been around for quite some time. Do you have any particular implementation in mind that I can download and run?

MarisaKirisame commented 4 years ago

That paper has runnable code. Haskell's AD package also has forward mode AD tower. http://www.fadbad.com/fadbad.html#General_introduction Also has higher order diff - it use forward over reverse but I assume it also has the simpler taylor series tower.

keithalewis commented 4 years ago

Haskell is not an option for me, but thanks for pointing this out. You also reminded me to take another look through www.Autodiff.org.

MarisaKirisame commented 4 years ago

If you had time, read the "evaluating derivative" book. I had seen numerous projects where ppl outside of AD start from scratch to build something, and after lots of work reinvent stuff already known. >90% of them reinvent stuff the book cover.