add AD engine

[Roger-luo]

not sure all the rules we need are forwarded, need to check with the PEPS example and forward whatever is missing

[codecov-commenter]

Codecov Report

Merging #14 (95cf719) into main (6a62b62) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #14   +/-   ##
=======================================
  Coverage   88.00%   88.00%           
=======================================
  Files           3        3           
  Lines         250      250           
=======================================
  Hits          220      220           
  Misses         30       30           

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6a62b62...95cf719. Read the comment docs.

[GiggleLiu]

A tried this AD engine a bit. I think the design is good in general. So I will merge it directly. But I do not understand why do we need TrackedArray, TrackedArrayTypes, TrackedRealArray, TrackedComplexArray? Isn't TrackedArray alone enough?

TODOs:

• If we want to support higher order gradients, we need to allow the gradient field being a TrackedArray, now it is forced to have the same type as input.
• We need to support differentiating a Tuple of array, because the AD of OMEinsum is defined on a tuple of array: https://github.com/under-Peter/OMEinsum.jl/blob/d5e22314eec16a6b8f59d9facb497fa9a7f15a0f/src/autodiff.jl#L32