CH4-Evaluating features for MNIST 3D-data

In previous parts, we have represented features as one long feature vector. For images, however, we often represent a m by n image as a (m,n) array, rather than a (mn,1) array (as the previous parts have done).

load_mnist_data function, which will read from the provided image files and populate a dictionary, with image and label vectors for each numerical digit from 0 to 9. These images are already shaped as (m,n) arrays.

"""
Returns a dictionary formatted as follows:
{
    0: {
        "images": [(m by n image), (m by n image), ...],
        "labels": [0, 0, ..., 0]
    },
    1: {...},
    ...
    9
}
Where labels range from 0 to 9 and (m, n) images are represented
by arrays of floats from 0 to 1
"""
mnist_data_all = hw3.load_mnist_data(range(10))

print('mnist_data_all loaded. shape of single images is', mnist_data_all[0]["images"][0].shape)

# HINT: change the [0] and [1] if you want to access different images
d0 = mnist_data_all[0]["images"]
d1 = mnist_data_all[1]["images"]
y0 = np.repeat(-1, len(d0)).reshape(1,-1)
y1 = np.repeat(1, len(d1)).reshape(1,-1)

# data goes into the feature computation functions
data = np.vstack((d0, d1))
# labels can directly go into the perceptron algorithm
labels = np.vstack((y0.T, y1.T)).T

6.1A) You may notice that some numbers (like 8) take up more horizontal space than others (like 1). We can compute a feature based on the average value in each row. Write a Python function that takes in a (m,n) array and returns a (m,1) array, where element i is the average value in row i.

def row_average_features(x):
#     """
#     This should either use or modify your code from the tutor questions.

#     @param x (n_samples,m,n) array with values in (0,1)
#     @return (m,n_samples) array where each entry is the average of a row
#     """
#     raise Exception("modify me!")
    return np.mean(x, axis=2,keepdims=False).T

6.1B) We can also compute a feature based on the average value in each column. Write a Python function that takes in a (m,n) array and returns a (n,1) array, where element j is the average value in column j.

def col_average_features(x):

#     """
#     This should either use or modify your code from the tutor questions.

#     @param x (n_samples,m,n) array with values in (0,1)
#     @return (n,n_samples) array where each entry is the average of a column
#     """
#     raise Exception("modify me!")
    return np.mean(x, axis=1,keepdims=False).T

6.1C) Finally, you may notice that some features are more "top heavy" while others are more "bottom heavy." Write a function that takes in a (m,n) array and returns a (2,1) array, where the first element is the average value in the top half of the image, and the second element is the average value in the bottom half of the image.

def top_bottom_features(x):
    p, m, n = x.shape
    ans = np.zeros((2,p))
    for i in range(p):
        ans[0,i]=np.mean(x[i,:m // 2,])
        ans[1,i]=np.mean(x[i,m // 2:,])
    return ans

Feature evaluation

For this section, we will be using T=50 with the base perceptron algorithm to train our classifier and 10-fold cross validation to evaluate our classifier. A function called get_classification_accuracy has already been implemented for you in code_for_hw3_part2 to compute the accuracy, given your selected data and labels.

def get_classification_accuracy(data, labels):
    """
    @param data (d,n) array
    @param labels (1,n) array
    """
    return xval_learning_alg(lambda data, labels: perceptron(data, labels, {"T": 50}), data, labels, 10)

6.2A) First we will find baseline accuracies using the raw 0-1 features. Convert each image into a (28*28, 1) vector for input into the perceptron algorithm. Hint: np.reshape may be helpful here. Run the perceptron on four tasks: 0 vs. 1, 2 vs. 4, 6 vs. 8, and 9 vs. 0. Solution: [0.975, 0.8641666666666665, 0.9479166666666667, 0.6470833333333333]

# HINT: change the [0] and [1] if you want to access different images
d0 = mnist_data_all[9]["images"]
d1 = mnist_data_all[0]["images"]

a0, a1, a2 = data.shape
data = data.reshape((a0, -1)).T
print(hw3.get_classification_accuracy(data, labels))

6.2B) Using the extracted features from above, run the perceptron algorithm on the set of 0 vs. 1 images. Solution: [0.48125, 0.6375, 0.48125] [row, column, top/bottom]

data = row_average_features(data)
data = col_average_features(data)
data = top_bottom_features(data)
print(hw3.get_classification_accuracy(data, labels))

Pin-Jiun / Machine-Learning-MIT

CH4-Evaluating features for MNIST 3D-data #9

Feature evaluation