Added support for Online Linear Regression

[kenluck2001]

This work is to create an online version of Linear regression as described in issue https://github.com/ddbourgin/numpy-ml/issues/70 . The implement the online version of the Ordinary least square using Matrix Inversion Lemma RLS, version 1 ( https://github.com/ddbourgin/numpy-ml/files/6536069/rls.pdf ).

Beware that there will be trade-off in some performance during the noisy nature of training on a sample at a time, in comparison to a batch. On the average, this should not be too significant.

To Do

Add unit test

• [x] Is the code you are submitting your own work? Yes
• [x] Did you properly attribute the authors of any code you referenced? Yes
• [x] Did you write unit tests for your new model?
• [x] Does your submission pass the unit tests?
• [ ] Did you write documentation for your new model?
• [ ] Have you formatted your code using the black deaults?

A sample for the API call is

olr = LinearRegression()
olr.fit(X, y)
olr.predict(rest-x)
# on new data just call update to modify beta field and update the model in an online manner
olr.update(x_new, y_new)

Note: no modification has happened to existing implementation as it is only active when update method is called.

[ddbourgin]

Awesome, @kenluck2001 - this is a welcome addition. I think the general strategy you followed here makes sense, but I'm going to go through it more closely soon. Thanks for the PR!

[ddbourgin]

Awesome, thanks. I took the liberty of refactoring the fit and update methods a bit to make things a bit cleaner (e.g., we now store the inverse of the data covariance matrix in fit so we can efficiently update it later on in the update method).

I also expanded the unit-test a little bit to verify correctness - instead of just checking the model R2, we new verify that the model coefficients and predictions match the gold standard implementation.

Have a look and let me know what you think. If everything looks good to you, I think this is ready to merge.

[kenluck2001]

Thanks @ddbourgin, it looks very nice to me. Feel free to merge the PR. I am very grateful for the great support.

[ddbourgin]

Awesome, thanks for the PR! Merged :) I'll try to take a look at the KMeans PR soon.