Added best-weight option to LinearRegression.predict API

[JHunterHobbs]

Fixes #14

This pull request adds a best weight option to the LinearRegression.predict() API so that the user can choose to use the best or last weights a prediction time. I've also added a print statement to inform when the weights are updated during training.

[rohansingh9001]

@JHunterHobbs I really apologise for not being able to reply earlier as I was busy in some other commitments. I also would like to apologise for any confusion caused due to some miscommunication.

To explain it very simply, by default we wish to save the weights only at the last step of the iteration. However, as may be useful in many applications we may want to save the weights where the model gives us the minimum loss.

Saving weights every time the loss betters itself in each case would be very redundant by default, as in the future when model sizes may grow to multiple megabytes or even gigabytes. We only want to deal with such scenarios when we really need to.

I request you to just accept a parameter like save_best with a default value of false (You may come up with a better name for the variable.) and save the model only when save_best is set to true. This will avoid potential newcomers to the code being confused by hundreds of saved files on their environment if they train their models for long enough. Let me explain it with some pseudocode.

if save_best and loss_in_current_epoch < best_loss_till_now:
    save_mode()
else:
    continue_training

Such an implementation will not require any special efforts and most probably require the addition of 2-3 lines of code.

What I believe @TarunTomar122 was referring to is the model checkpoint feature of TensorFlow.

It lets us save the model at certain intervals that we can define, one of them being when the model achieves the best loss during the current training session. However, it is not used by default. It must be added by the user if he/she wishes to use it.

Also, there might be some discrepancy in how the issue is described. I will request the author of the issue to correct it ASAP. I will also go through all issues to see if they are up to the mark and describe everything correctly. Thanks for letting us know and contributing to our repository😃.

[JHunterHobbs]

@rohansingh9001 - So you are wanting to save a snapshot of the model ever time the training finds a new minimum loss? Won't that result in many unnecessary snapshots/files being saved during training? Especially at first when the weights are far from any sort of minimum and still attempting to follow the gradient.

[rohansingh9001]

@JHunterHobbs you got it right. Only save it when the model gives us the minimum loss and another save must be made at the final epoch.

I realised maybe my pseudocode might save it at each step where the model performs better than till now.

However if you are still confused, we can merge your PR and I can later add the parameter myself or maybe someone else might work on that. I hope that should resolve the issue quickly.

[JHunterHobbs]

I've added the flag to the fit function to optionally save the model when a new minimum is found during training. I've remove the flag for the predict and predict will only use the last weights of training. I've differentiated each model by adding a timestamp to the filename. The best models will be saved as model_best_03-10-2020_22:04:48.rob and the final model (save after all epoch during training) will be saved as model_final_{timestamp}.rob.

Please have a look whenever you have some time. Thanks!