Broadcasting Support for N-Dimensional Array

[karanchahal]

In this project, we use a ND Array to represent the underlying data of a Tensor. The Matrix class represents the ND array.

As of now, we have little to no support for broadcasting as one would expect from ND arrays (like numpy arrays).

So for now we can perform various ops like addition, multiplication, division etc on elements of the same size, with ops being applied between 2 elements that are in the same position. However, that needs to change as when we move on to implement ops like softmax, we need support for applying operations across unequal size tensors.

For example, in the softmax operation, we have a sum varibale which is the sum of all elements in the data. And the softmax procedure dictate that we divide each element by this sum.

This is trivially implementable if we have a single dimension array. DIvision with a scalar solves our problem.

However, the problem becomes complicated when we have a mini batch of examples over which we want to apply softmax across an axis. Dividing the sum needs some form of broadcasting.

This issue will track the rules of broadcasting for our project. Most likely we want broadcasting to be very intuitive and similar to numpy's functionality.

[karanchahal]

Prelim broadcasting support will come through the pull request for the Softmax layer.