Closed okumuskaan closed 1 year ago
Hi @okumuskaan, thanks for this PR, lots of work covered there! There are a few things that will need to be fixed before I can approve the PR. Here is a list:
sin
for Sin
) as well as the non standard arguments specific to certain classes such as base
for Exp/Log
. Also give in a Notes section (Numpy docstring style) an expression of the derivative of the ufunc and the values of the lipschitz
/diff_lipschitz
constants when known (so that the user knows if the latter are available). Make sure you use LaTeX from within the ..math ::
of :math:
balises from Sphinx. Finally, add a See Also (Numpy docstring style) section where you point to classes from within the same family (e.g. cos points to other trigonometric functions). ufunc.py
make a docstring and explain/provide examples on how to use the classes/functions of the module in practice (i.e. Sin(shape) * Op
vssin(op)
for example). enforce_precision
, otherwise the input arr
used to create Jacobian matrices will not be coerced to the requested type. Similarly self._base
should be coerced with pycsou.runtime.coerce
in Exp/Log.apply/adjoint
otherwise it will recast the whole computation to the wrong precision.Prod
can be simplified to:
@pycrt.enforce_precision(i="arr")
def grad(self, arr: pyct.NDArray) -> pyct.NDArray:
xp = pycu.get_array_module(arr)
g = xp.repeat(x[..., None, :], x.shape[-1], axis=0)
e = xp.eye(*g.shape[-2:]).broadcast_to(g.shape).astype(bool)
g[e] = pycrt.coerce(1.)
return xp.prod(g, axis=-1)
With this implementation you do not need explicit handling of zeros as you do in _grad_row
.
Cumprod/sum
the argument axis
should come with an argument argshape
: by convention, Pycsou always acts on the last axis of input arrays so there is theoretically no need to provide an axis
keyword. The latter can however be useful if the stacks (...,N)
of input arrays can be reinterpreted as stacks of flattened multidimensional inputs (..., *argshape)
. We should discuss that at the next dev meeting (@SepandKashani). Cumprod.jacobian
using the simplifiedProd.jacobian
implementation above.Clip
should be a DiffMap
(it is almost everywhere differentiable, so numerically we can consider it is differentiable just like Abs
). Provide its derivative (1 over [a_min, a_max] and 0 elsewhere) and lipschitz
/diff_lipschitz
constants. Same for Sign
.ReLU
via Clip
with a_min, a_max = 0, None
.GELU.apply
is not module agnostic: you use erf
from Scipy which only works with Numpy. Make sure your implementation works for CuPy/Dask too. StopCriterion_LSQR
from this PR please .test_ufunc.py
to reflect all the changes above. That's it, let me know if anything is unclear, I'd be happy to help. And thanks again for the much appreciated work.
Summary of Last Commits
operator/map/__init__.py
.ufunc
module and its functions are implemented.Cumprod
, Sum
are deleted.Clip
, Sign
functions are converted from Map
to DiffMap
.ReLU
is implemented via Clip
.GELU
is made to be module agnostic.test_ufunc.py
is adapted.Closed by mistake.