Optimization Techiniques for Machine Learning project work

Stochastic Gradient Descent with Momentum and Line Searches. Implementation, hyper-parameters tuning and efficiency testing of five variants of the Minibatch Gradient Descent algorithm on a benchmark of six real-world datasets, performing binary classification with Logistic Regression.

• SGD with fixed learning rate
• SGD with decreasing learning rate
• SGD with momentum (SGDM)
• SGD with Armijo line search for optimal learning rate during iterations
• SGD with momentum correction and restart, and Armijo line search (MSL-SGDM-C/R)