When using dynamic dispatch or when defining custom derivatives, the custom derivative itself may no longer be differentiable.
Currently the implementation treats all custom derivative functions as differentiable and apply checks on them to ensure they are differentiable. This can lead to unnecessary restrictions when higher order differentiation isn't needed.
We should strengthen our type checking to allow defining non differentiable derivatives. The primal function then has a [Differentiable(1)] property, meaning that it is 1-order differentiable. We also need to implement the propagation rules to ensure that any functions that calls an n-order differentiable function become an n-order differentiable function, and enforce that an interface method must have a finite differentiation order.
When using dynamic dispatch or when defining custom derivatives, the custom derivative itself may no longer be differentiable.
Currently the implementation treats all custom derivative functions as differentiable and apply checks on them to ensure they are differentiable. This can lead to unnecessary restrictions when higher order differentiation isn't needed.
We should strengthen our type checking to allow defining non differentiable derivatives. The primal function then has a
[Differentiable(1)]
property, meaning that it is 1-order differentiable. We also need to implement the propagation rules to ensure that any functions that calls an n-order differentiable function become an n-order differentiable function, and enforce that an interface method must have a finite differentiation order.