Hands-on Machine Learning Book

[Starsa]

Read chapter daily. Write brief summary and list any questions below

• [x] Monday
• [x] Tuesday
• [x] Wednesday
• [x] Thursday
• [x] Friday
• [ ] Saturday/Sunday

[Starsa]

Monday:

Chapter 7- Random Forest & Ensemble Learning

• Voting Classifiers aggregates the predictions for multiple models to predict the class with the most votes
  • law of large numbers
  • only for independent classifiers, can have uncorrelated errors
• Bagging and Pasting
  • use the same training algorithm with a different subset of the data to vote
  • Bagging is with replacement
  • Pasting is without replacement
• Random forest
  • is an ensemble method using multiple decision trees
  • trained via bagging
  • Feature importance
  • looking at tree nodes that reduce impurity
• Boosting
  • combines several weak learners into strong learner
  • Gradient boosting
  • tries to fit the predictor to the residual error made by previous error
• Stacking
  • train a model to aggregate predictions

No Questions

[Starsa]

Tuesday:

Chapter 8- Dimensionality Reduction

• The curse of Dimensionality
  • more dimensions the training set has, the greater risk of overfitting
• Approaches to dimensionality reduction
  • Projection - project every training instance perpendicularly onto subspace for 2d
  • Manifold- A 2D shape that can be bent and twisted in a higher dimensional hyperplane
  • PCA
  • identifies the hyperplane that lies closest to the data and then projects the data onto it
  • it accounts for the largest amount of variance
  • USE SVD to decompose matrix into separate matricies
  • find explained variance ratio indicates the data's variance along each principal component
  • Choosing the right number of dimensions
    • compute PCA without reducing dimensions, then compute min num of dimensions reqed to preserve 95% variance
    • plot explained variance for elbow
  • Kernel trick- mathematical technique to map instances into a very high dimensional space
    • use grid search

[Starsa]

Wednesday:

Chapter 9: Unsupervised Learning Techniques

• Clustering
  • customer segmentation
  • data analysis
  • dimensionality reduction
  • anomaly detection (outlier detection)
  • search engines (image recognition)
  • K-Means
    • use an elbow plot for ideal k or silhouette score (mean silhouette coefficient)
  • DBSCAN
    • defines clusters as continuous regions of high density
    • robust to outliers
  • Agglomerative clustering
  • Mean shift
  • spectral clustering
• Gaussian Mixtures
  • AIC and BIC
  • both penalize models that have more parameters to learn and reward models that fit the data well

[Starsa]

Thursday:

Chapter 10: Introduction to Artificial Neural Networks with Keras

• ANN's -Artifical Neural Network inspired by neurons in our brain
• Perceptrons
  • Based on TLU (threshold logic unit)
  • Composed of a single layer TLU of TLUs with each TLU connected to all the imputs
• MLP (Multi Layer Perceptron)
  • Eliminates some of the limitations of perception by stacking multiple Perceptrons
  • composed of:
  • One passthrough input layer
  • one or more layers of TLU's (called hidden layers)
  • One final layer of TLU's
  • DNN (Deep Neural Network)
  • An ANN containing deep stack of hidden layers
  • Backpropogation
  • Gradient Decent computing gradients automatically (autodiff) in two passes (fwd, bkwd)
  • computes the error with every single model parameter
  • Epoch
    • Each time all observations have been sent through the network
    • Training neural networks involves multiple epochs
  • "For each training instance, the backpropogation algorithm first makes a prediction (forward-pass) and measures the error, then goes through each layer in reverse to measure error contribution from each connection (reverse pass), and finally tweaks the connection wright to reduce the error (Gradient Descent step)
• Keras:
  • Deep learning libraries:
  • TensorFlow
  • Microsoft Cognitive Toolkit (CNTK)
  • Theano
  • Facebooks (pytorch)