dlsyscourse / hw1

7 stars 27 forks source link

Add test that checks iterating through broadcast dims from the rightmost dim #9

Closed LiableFish closed 1 year ago

LiableFish commented 2 years ago

As numpy docs state

When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimensions and works its way left

But no of current tests (either local or mugrade one) actually checks that you're trying to find broadcast dims iterating from the right and not from left during backward pass.

I was able to pass all the tests in HW1, but failed on one of the Linear module's test in HW2 just because I'm iterating from the left. Particularly I'm using zip_longest to match the input dims and required dims. So for theinput_dim = (5,) and required_dim = (1, 5) I'm dealing with (5, None) and (1, 5) instead of (None, 5) and (1, 5). That's why I failed on the test. And that's why I suggest to add it in the HW1 :)

navalnica commented 2 years ago

I've also added similar test for broadcast_to method:

def test_broadcast_to_backward_my():
    gradient_check(ndl.broadcast_to, ndl.Tensor(np.random.randn(3, 1, 5)), shape=(2, 3, 4, 5))

it combines checks for (1) different number of dimensions and for (2) broadcast over existing dimension

LiableFish commented 2 years ago
gradient_check(ndl.broadcast_to, ndl.Tensor(np.random.randn(3, 1, 5)), shape=(2, 3, 4, 5))

Added it too, good point