Open ChrisRackauckas opened 2 years ago
I did toy around with FFT Plans at an earlier point (#11 #14) but then put it off since it turned out to be too much hassle for me at that time - does that cover something different?
Nope, that's the same thing. To avoid those issues, a direct rrule will be needed.
Similar to the
FastDense
optimizations in https://github.com/SciML/DiffEqFlux.jl/pull/671, this library can definitely benefit from having pre-cached versions of the operations since the neural networks are generally small. In addition, theplan_fft
part could be cached and reused for subsequent calls. Given the amount of reuse, direct control of the planning could be helpful:Note that the precaching only removes allocations in cases with a single forward before reverse. A separate pointer bumping method would be necessary to precache a whole batch of test inputs, if multiple batches are used in one loss equation.