Quansight-Labs / udiff

Automatic differentiation with uarray/unumpy.
BSD 3-Clause "New" or "Revised" License
16 stars 11 forks source link

add np.sign & np.abs #6

Closed sangyx closed 4 years ago

sangyx commented 4 years ago

Hi, I found a bug in np.positive and np.negative. The diff calculate in the source is incorrect:

register_diff(np.positive, lambda x: +x[1])
register_diff(np.negative, lambda x: -x[1])

If we suppose that x = [1, -1], then the y = np.positive(x)=[1, 1]. We can see that y = x if x > 0 and y=-x if x < 0. So the diff of y should be y=sign(x)=[1, -1]

hameerabbasi commented 4 years ago

Hello. I think you’re confusing what np.positive and np.negative do. They correspond to +x and -x, regardless of the value of x. So np.positive(-5) = -5 and np.negative(-5) = +5. Your derivative for np.sign is also slightly incorrect, unfortunately, as it’s undefined at x == 0. So it should be np.broadcast_to(np.where(x[0] == 0, float('nan'), 0), x[1].shape).

The second part is due to the product rule where the input is a function of the input. So you apply the chain rule f’(x) * d/dx g(f(x)) where f(x) is x[0], g(x) is np.sign(x) in this case, and f’(x) is x[1] here.

hameerabbasi commented 4 years ago

What you’re suggesting is true for np.abs, however.

sangyx commented 4 years ago

Sorry for that. I can't guess the usage of the two functions and I should open an issue for this before the pr. I will try to complete the np.abs. But I think there is another problem. np.exp2 calculates 2**p for all p in the input array. But the code to calculate the derivative for it is :

register_diff(np.exp2, lambda x: x[1] * np.log(2) * np.exp(x[0]))

It should be:

register_diff(np.exp2, lambda x: x[1] * np.log(2) * np.power(2, x[0]))
hameerabbasi commented 4 years ago

Ah, yes. That’s a mistake on my part, same for exp10. You can go ahead and fix that. I’d replace np.power(2, x) with np.exp2(x), but the rest seems good.

sangyx commented 4 years ago

Hi, when I complete the np.abs as:

register_diff(np.sign, lambda x: np.broadcast_to(np.where(x[0] == 0, float('nan'), 0), x[1].shape))
register_diff(np.absolute, lambda x: x[1] * np.where(np.sign(x[0]) == 0, float('nan'), np.sign(x[0])))

The diff of np.abs is fine. But the diff of np.sign encounter a problem:

PS F:\OneDrive - sjtu.edu.cn\Project\GSOC\udiff\src> python .\test.py
<DiffArray, name=unbound, arr=
[2. 3. 0.]
>
<DiffArray, name=unbound, arr=
[ 1. -1. nan]
>
Traceback (most recent call last):
  File ".\test.py", line 12, in <module>
    y = np.sign(x)
  File "F:\OneDrive - sjtu.edu.cn\Project\GSOC\udiff\src\udiff\_uarray_plug.py", line 65, in __ua_function__
    diff_arr = global_registry[a[0]](*a[1:], **kw)
  File "F:\OneDrive - sjtu.edu.cn\Project\GSOC\udiff\src\udiff\_builtin_diffs.py", line 32, in <lambda>
    register_diff(np.sign, lambda x: np.broadcast_to(np.where(x[0] == 0, float('nan'), 0), x[1].shape))
  File "D:\Python\lib\site-packages\unumpy\_multimethods.py", line 105, in f
    return globals()[name](self, other)
  File "F:\OneDrive - sjtu.edu.cn\Project\GSOC\udiff\src\udiff\_uarray_plug.py", line 65, in __ua_function__
    diff_arr = global_registry[a[0]](*a[1:], **kw)
KeyError: <ufunc 'equal'>

But I don't know how to complete the diff of np.equal.

hameerabbasi commented 4 years ago

Hmm. Try x[0].arr.

sangyx commented 4 years ago

Thx, the pr have committed.

hameerabbasi commented 4 years ago

Thanks @sangyx. This is in!