Machine learning from scratch

Numpy-ml from scratch. This repo aims to help myself/people understand the math behind machine learning algorithms and I will try to make the computation as efficient as possible

Implementations

Supervised Learning

• Decision Tree
• Grdient Boosting Tree
• Logistic Regression
• Linear Regression
• Elastic Net
• Ridge Regression
• Lasso Regression
• Support Vector Machine
• Xgboost

Deep Learning

• Neural Network

• Layers

  • Activation Layer
  • Batch Normalization Layer
  • Dropout Layer
  • Fully Connected Layer
  • Embedding Layer
  • RNN Layer: many-to-one
  • LSTM ayer: many-to-one
  • Bidirectional LSTM

• Loss Functions

  • Cross Entropy
  • Loss for VAE
  • BinomialDeviance
  • Noise Contrastive Estimation

• Optimizer

  • SGD with momentum
  • RMSprop
  • Adagrad
  • Adadelta
  • Adam

• Schedulers

  • CosineAnnealingLR
  • CosineAnnealingWarmRestarts

• Models

  • word2vec
  • LSTM many to many

Unsupervised Learning

• Generative Adversarial Network

Examples

SVM

Polynomial Lasso Regression

Decision Tree for Classification

Decision Tree for Regression

Xgboost

deep learning

The result of unit test for different parts of deep learning

The warning message is due to the bug of Tensorflow

unit test page