TODOs about AD engines - Githubissues

A tried this AD engine a bit. I think the design is good in general. So I will merge it directly. But I do not understand

Why do we need TrackedArray, TrackedArrayTypes, TrackedRealArray, TrackedComplexArray? Isn't TrackedArray alone enough?

TODOs:

[ ] If we want to support higher order gradients, we need to allow the gradient field being a TrackedArray, now it is forced to have the same type as input. Also, accumulate_grad! is an inplace function, not sure it can be differentiated.
[x] We need to support differentiating a Tuple of array, because the AD of OMEinsum is defined on a tuple of array: https://github.com/under-Peter/OMEinsum.jl/blob/d5e22314eec16a6b8f59d9facb497fa9a7f15a0f/src/autodiff.jl#L32