assignment1-5 - Githubissues

JinwoongKim commented 6 years ago

for lr in learning_rates:
    for reg in regularization_strengths:
        svm = LinearSVM()
        svm.train(X_train_feats, y_train, learning_rate=lr, reg=reg, num_iters=1500, verbose=False)
        y_train_pred = svm.predict(X_train_feats)
        y_val_pred = svm.predict(X_val_feats)
        train_accuracy = np.mean(y_train == y_train_pred)
        val_accuracy = np.mean(y_val == y_val_pred)
        print('train accuracy : ',train_accuracy)
        print('val accuracy : ', val_accuracy)
        results[(lr,reg)] = (train_accuracy, val_accuracy)
        if val_accuracy > best_val:
            best_val = val_accuracy
            best_svm = svm
            print('best_val : ',best_val)
            print('best_svm : ',best_svm)

JinwoongKim commented 6 years ago

Previous result in assign 1-2 was 0.39 !!

JinwoongKim commented 6 years ago

input_dim = X_train_feats.shape[1]
hidden_dim = 500
num_classes = 10

net = TwoLayerNet(input_dim, hidden_dim, num_classes)
best_net = None

################################################################################
# TODO: Train a two-layer neural network on image features. You may want to    #
# cross-validate various parameters as in previous sections. Store your best   #
# model in the best_net variable.                                              #
################################################################################
best_val_acc = -1

learning_rates = [0.001, 0.01, 0.1]
learning_rate_decay = [0.6, 0.7, 0.75, 0.8,0.85,0.9, 0.95]
regularization_strengths = [0.2, 0.25, 0.3, 0.35, 0.40]

for lr in learning_rates:
    for lr_d in learning_rate_decay:
        for reg in regularization_strengths:
            # Train the network
            net = TwoLayerNet(input_dim, hidden_dim, num_classes)

            stats = net.train(X_train_feats, y_train, X_val_feats, y_val,
            num_iters=1000, batch_size=200,
            learning_rate=lr, learning_rate_decay=lr_d,
            reg=reg, verbose=False)

            # Predict on the validation set
            val_acc = (net.predict(X_val_feats) == y_val).mean()
            print('Validation accuracy: ', val_acc)

            if val_acc > best_val_acc:
                best_val_acc = val_acc
                best_net = net
                print('best_val : ',best_val_acc)
                print('lr lrd reg : ',lr, lr_d, reg)

JinwoongKim commented 6 years ago

Validation accuracy: 0.263 lr lrd reg : 0.1 0.3 0.2 Current best accuracy: 0.263 lr lrd reg : 0.1 0.3 0.2 Validation accuracy: 0.266 lr lrd reg : 0.1 0.3 0.25 Current best accuracy: 0.266 lr lrd reg : 0.1 0.3 0.25 Validation accuracy: 0.233 lr lrd reg : 0.1 0.3 0.3 Validation accuracy: 0.254 lr lrd reg : 0.1 0.3 0.35 Validation accuracy: 0.238 lr lrd reg : 0.1 0.3 0.4 Validation accuracy: 0.229 lr lrd reg : 0.1 0.35 0.2 Validation accuracy: 0.228 lr lrd reg : 0.1 0.35 0.25 Validation accuracy: 0.217 lr lrd reg : 0.1 0.35 0.3 Validation accuracy: 0.216 lr lrd reg : 0.1 0.35 0.35 Validation accuracy: 0.206 lr lrd reg : 0.1 0.35 0.4 Validation accuracy: 0.211 lr lrd reg : 0.1 0.4 0.2 Validation accuracy: 0.21 lr lrd reg : 0.1 0.4 0.25 Validation accuracy: 0.202 lr lrd reg : 0.1 0.4 0.3 Validation accuracy: 0.204 lr lrd reg : 0.1 0.4 0.35 Validation accuracy: 0.2 lr lrd reg : 0.1 0.4 0.4 Validation accuracy: 0.202 lr lrd reg : 0.1 0.45 0.2 Validation accuracy: 0.205 lr lrd reg : 0.1 0.45 0.25 Validation accuracy: 0.196 lr lrd reg : 0.1 0.45 0.3 Validation accuracy: 0.192 lr lrd reg : 0.1 0.45 0.35 Validation accuracy: 0.192 lr lrd reg : 0.1 0.45 0.4 Validation accuracy: 0.196 lr lrd reg : 0.1 0.5 0.2 Validation accuracy: 0.196 lr lrd reg : 0.1 0.5 0.25 Validation accuracy: 0.192 lr lrd reg : 0.1 0.5 0.3 Validation accuracy: 0.189 lr lrd reg : 0.1 0.5 0.35 Validation accuracy: 0.161 lr lrd reg : 0.1 0.5 0.4 Validation accuracy: 0.196 lr lrd reg : 0.1 0.55 0.2 Validation accuracy: 0.196 lr lrd reg : 0.1 0.55 0.25 Validation accuracy: 0.189 lr lrd reg : 0.1 0.55 0.3 Validation accuracy: 0.189 lr lrd reg : 0.1 0.55 0.35 Validation accuracy: 0.189 lr lrd reg : 0.1 0.55 0.4 Validation accuracy: 0.199 lr lrd reg : 0.2 0.3 0.2 Validation accuracy: 0.202 lr lrd reg : 0.2 0.3 0.25 Validation accuracy: 0.194 lr lrd reg : 0.2 0.3 0.3 Validation accuracy: 0.193 lr lrd reg : 0.2 0.3 0.35 Validation accuracy: 0.193 lr lrd reg : 0.2 0.3 0.4 Validation accuracy: 0.196 lr lrd reg : 0.2 0.35 0.2 Validation accuracy: 0.203 lr lrd reg : 0.2 0.35 0.25 Validation accuracy: 0.189 lr lrd reg : 0.2 0.35 0.3 Validation accuracy: 0.189 lr lrd reg : 0.2 0.35 0.35 Validation accuracy: 0.193 lr lrd reg : 0.2 0.35 0.4 Validation accuracy: 0.195 lr lrd reg : 0.2 0.4 0.2 Validation accuracy: 0.192 lr lrd reg : 0.2 0.4 0.25 Validation accuracy: 0.188 lr lrd reg : 0.2 0.4 0.3 Validation accuracy: 0.188 lr lrd reg : 0.2 0.4 0.35 Validation accuracy: 0.148 lr lrd reg : 0.2 0.4 0.4 Validation accuracy: 0.193 lr lrd reg : 0.2 0.45 0.2 Validation accuracy: 0.195 lr lrd reg : 0.2 0.45 0.25 Validation accuracy: 0.188 lr lrd reg : 0.2 0.45 0.3 Validation accuracy: 0.202 lr lrd reg : 0.2 0.45 0.35 Validation accuracy: 0.071 lr lrd reg : 0.2 0.45 0.4 Validation accuracy: 0.192 lr lrd reg : 0.2 0.5 0.2 Validation accuracy: 0.196 lr lrd reg : 0.2 0.5 0.25 Validation accuracy: 0.187 lr lrd reg : 0.2 0.5 0.3 Validation accuracy: 0.183 lr lrd reg : 0.2 0.5 0.35 Validation accuracy: 0.128 lr lrd reg : 0.2 0.5 0.4 Validation accuracy: 0.192 lr lrd reg : 0.2 0.55 0.2 Validation accuracy: 0.195 lr lrd reg : 0.2 0.55 0.25 Validation accuracy: 0.187 lr lrd reg : 0.2 0.55 0.3 Validation accuracy: 0.137 lr lrd reg : 0.2 0.55 0.35 Validation accuracy: 0.112 lr lrd reg : 0.2 0.55 0.4 Validation accuracy: 0.194 lr lrd reg : 0.3 0.3 0.2 Validation accuracy: 0.195 lr lrd reg : 0.3 0.3 0.25 Validation accuracy: 0.189 lr lrd reg : 0.3 0.3 0.3 Validation accuracy: 0.188 lr lrd reg : 0.3 0.3 0.35 Validation accuracy: 0.194 lr lrd reg : 0.3 0.3 0.4 Validation accuracy: 0.193 lr lrd reg : 0.3 0.35 0.2 Validation accuracy: 0.195 lr lrd reg : 0.3 0.35 0.25 Validation accuracy: 0.188 lr lrd reg : 0.3 0.35 0.3 Validation accuracy: 0.204 lr lrd reg : 0.3 0.35 0.35 Validation accuracy: 0.082 lr lrd reg : 0.3 0.35 0.4 Validation accuracy: 0.192 lr lrd reg : 0.3 0.4 0.2 Validation accuracy: 0.195 lr lrd reg : 0.3 0.4 0.25 Validation accuracy: 0.186 lr lrd reg : 0.3 0.4 0.3 Validation accuracy: 0.182 lr lrd reg : 0.3 0.4 0.35 Validation accuracy: 0.167 lr lrd reg : 0.3 0.4 0.4 Validation accuracy: 0.194 lr lrd reg : 0.3 0.45 0.2 Validation accuracy: 0.193 lr lrd reg : 0.3 0.45 0.25 Validation accuracy: 0.188 lr lrd reg : 0.3 0.45 0.3 Validation accuracy: 0.215 lr lrd reg : 0.3 0.45 0.35 Validation accuracy: 0.117 lr lrd reg : 0.3 0.45 0.4 Validation accuracy: 0.192 lr lrd reg : 0.3 0.5 0.2 Validation accuracy: 0.195 lr lrd reg : 0.3 0.5 0.25 Validation accuracy: 0.165 lr lrd reg : 0.3 0.5 0.3 Validation accuracy: 0.139 lr lrd reg : 0.3 0.5 0.35 Validation accuracy: 0.151 lr lrd reg : 0.3 0.5 0.4 Validation accuracy: 0.193 lr lrd reg : 0.3 0.55 0.2 Validation accuracy: 0.196 lr lrd reg : 0.3 0.55 0.25 Validation accuracy: 0.19 lr lrd reg : 0.3 0.55 0.3 Validation accuracy: 0.182 lr lrd reg : 0.3 0.55 0.35 Validation accuracy: 0.198 lr lrd reg : 0.3 0.55 0.4 best_val, set : 0.266 (0.1, 0.3, 0.25)

JinwoongKim commented 6 years ago

Validation accuracy: 0.299 lr lrd reg : 0.01 0.3 0.2 Current best accuracy: 0.299 lr lrd reg : 0.01 0.3 0.2 Validation accuracy: 0.326 lr lrd reg : 0.01 0.3 0.25 Current best accuracy: 0.326 lr lrd reg : 0.01 0.3 0.25 Validation accuracy: 0.338 lr lrd reg : 0.01 0.3 0.3 Current best accuracy: 0.338 lr lrd reg : 0.01 0.3 0.3 Validation accuracy: 0.305 lr lrd reg : 0.01 0.3 0.35 Validation accuracy: 0.303 lr lrd reg : 0.01 0.3 0.4 Validation accuracy: 0.318 lr lrd reg : 0.01 0.35 0.2 Validation accuracy: 0.343 lr lrd reg : 0.01 0.35 0.25 Current best accuracy: 0.343 lr lrd reg : 0.01 0.35 0.25 Validation accuracy: 0.32 lr lrd reg : 0.01 0.35 0.3 Validation accuracy: 0.332 lr lrd reg : 0.01 0.35 0.35 Validation accuracy: 0.348 lr lrd reg : 0.01 0.35 0.4 Current best accuracy: 0.348 lr lrd reg : 0.01 0.35 0.4 Validation accuracy: 0.36 lr lrd reg : 0.01 0.4 0.2 Current best accuracy: 0.36 lr lrd reg : 0.01 0.4 0.2 Validation accuracy: 0.345 lr lrd reg : 0.01 0.4 0.25 Validation accuracy: 0.33 lr lrd reg : 0.01 0.4 0.3 Validation accuracy: 0.356 lr lrd reg : 0.01 0.4 0.35 Validation accuracy: 0.327 lr lrd reg : 0.01 0.4 0.4 Validation accuracy: 0.366 lr lrd reg : 0.01 0.45 0.2 Current best accuracy: 0.366 lr lrd reg : 0.01 0.45 0.2 Validation accuracy: 0.362 lr lrd reg : 0.01 0.45 0.25 Validation accuracy: 0.379 lr lrd reg : 0.01 0.45 0.3 Current best accuracy: 0.379 lr lrd reg : 0.01 0.45 0.3 Validation accuracy: 0.352 lr lrd reg : 0.01 0.45 0.35 Validation accuracy: 0.345 lr lrd reg : 0.01 0.45 0.4 Validation accuracy: 0.364 lr lrd reg : 0.01 0.5 0.2 Validation accuracy: 0.38 lr lrd reg : 0.01 0.5 0.25 Current best accuracy: 0.38 lr lrd reg : 0.01 0.5 0.25 Validation accuracy: 0.376 lr lrd reg : 0.01 0.5 0.3 Validation accuracy: 0.352 lr lrd reg : 0.01 0.5 0.35 Validation accuracy: 0.375 lr lrd reg : 0.01 0.5 0.4 Validation accuracy: 0.394 lr lrd reg : 0.01 0.55 0.2 Current best accuracy: 0.394 lr lrd reg : 0.01 0.55 0.2 Validation accuracy: 0.374 lr lrd reg : 0.01 0.55 0.25 Validation accuracy: 0.367 lr lrd reg : 0.01 0.55 0.3 Validation accuracy: 0.376 lr lrd reg : 0.01 0.55 0.35 Validation accuracy: 0.359 lr lrd reg : 0.01 0.55 0.4 Validation accuracy: 0.383 lr lrd reg : 0.01 0.6 0.2 Validation accuracy: 0.367 lr lrd reg : 0.01 0.6 0.25 Validation accuracy: 0.367 lr lrd reg : 0.01 0.6 0.3 Validation accuracy: 0.375 lr lrd reg : 0.01 0.6 0.35 Validation accuracy: 0.387 lr lrd reg : 0.01 0.6 0.4 Validation accuracy: 0.374 lr lrd reg : 0.01 0.65 0.2 Validation accuracy: 0.405 lr lrd reg : 0.01 0.65 0.25 Current best accuracy: 0.405 lr lrd reg : 0.01 0.65 0.25 Validation accuracy: 0.352 lr lrd reg : 0.01 0.65 0.3 Validation accuracy: 0.366 lr lrd reg : 0.01 0.65 0.35 Validation accuracy: 0.368 lr lrd reg : 0.01 0.65 0.4 Validation accuracy: 0.362 lr lrd reg : 0.01 0.7 0.2 Validation accuracy: 0.374 lr lrd reg : 0.01 0.7 0.25 Validation accuracy: 0.372 lr lrd reg : 0.01 0.7 0.3 Validation accuracy: 0.354 lr lrd reg : 0.01 0.7 0.35 Validation accuracy: 0.362 lr lrd reg : 0.01 0.7 0.4 Validation accuracy: 0.368 lr lrd reg : 0.01 0.75 0.2 Validation accuracy: 0.34 lr lrd reg : 0.01 0.75 0.25 Validation accuracy: 0.374 lr lrd reg : 0.01 0.75 0.3 Validation accuracy: 0.346 lr lrd reg : 0.01 0.75 0.35 Validation accuracy: 0.381 lr lrd reg : 0.01 0.75 0.4 Validation accuracy: 0.374 lr lrd reg : 0.05 0.3 0.2 Validation accuracy: 0.356 lr lrd reg : 0.05 0.3 0.25 Validation accuracy: 0.36 lr lrd reg : 0.05 0.3 0.3 Validation accuracy: 0.358 lr lrd reg : 0.05 0.3 0.35 Validation accuracy: 0.333 lr lrd reg : 0.05 0.3 0.4 Validation accuracy: 0.324 lr lrd reg : 0.05 0.35 0.2 Validation accuracy: 0.34 lr lrd reg : 0.05 0.35 0.25 Validation accuracy: 0.335 lr lrd reg : 0.05 0.35 0.3 Validation accuracy: 0.34 lr lrd reg : 0.05 0.35 0.35 Validation accuracy: 0.326 lr lrd reg : 0.05 0.35 0.4 Validation accuracy: 0.311 lr lrd reg : 0.05 0.4 0.2 Validation accuracy: 0.31 lr lrd reg : 0.05 0.4 0.25 Validation accuracy: 0.304 lr lrd reg : 0.05 0.4 0.3 Validation accuracy: 0.308 lr lrd reg : 0.05 0.4 0.35 Validation accuracy: 0.289 lr lrd reg : 0.05 0.4 0.4 Validation accuracy: 0.268 lr lrd reg : 0.05 0.45 0.2 Validation accuracy: 0.271 lr lrd reg : 0.05 0.45 0.25 Validation accuracy: 0.264 lr lrd reg : 0.05 0.45 0.3 Validation accuracy: 0.259 lr lrd reg : 0.05 0.45 0.35 Validation accuracy: 0.248 lr lrd reg : 0.05 0.45 0.4 Validation accuracy: 0.242 lr lrd reg : 0.05 0.5 0.2 Validation accuracy: 0.246 lr lrd reg : 0.05 0.5 0.25 Validation accuracy: 0.233 lr lrd reg : 0.05 0.5 0.3 Validation accuracy: 0.236 lr lrd reg : 0.05 0.5 0.35 Validation accuracy: 0.217 lr lrd reg : 0.05 0.5 0.4 Validation accuracy: 0.226 lr lrd reg : 0.05 0.55 0.2 Validation accuracy: 0.229 lr lrd reg : 0.05 0.55 0.25 Validation accuracy: 0.211 lr lrd reg : 0.05 0.55 0.3 Validation accuracy: 0.212 lr lrd reg : 0.05 0.55 0.35 Validation accuracy: 0.209 lr lrd reg : 0.05 0.55 0.4 Validation accuracy: 0.211 lr lrd reg : 0.05 0.6 0.2 Validation accuracy: 0.213 lr lrd reg : 0.05 0.6 0.25 Validation accuracy: 0.21 lr lrd reg : 0.05 0.6 0.3 Validation accuracy: 0.196 lr lrd reg : 0.05 0.6 0.35 Validation accuracy: 0.19 lr lrd reg : 0.05 0.6 0.4 Validation accuracy: 0.207 lr lrd reg : 0.05 0.65 0.2 Validation accuracy: 0.205 lr lrd reg : 0.05 0.65 0.25 Validation accuracy: 0.203 lr lrd reg : 0.05 0.65 0.3 Validation accuracy: 0.214 lr lrd reg : 0.05 0.65 0.35 Validation accuracy: 0.193 lr lrd reg : 0.05 0.65 0.4 Validation accuracy: 0.199 lr lrd reg : 0.05 0.7 0.2 Validation accuracy: 0.201 lr lrd reg : 0.05 0.7 0.25 Validation accuracy: 0.193 lr lrd reg : 0.05 0.7 0.3 Validation accuracy: 0.19 lr lrd reg : 0.05 0.7 0.35 Validation accuracy: 0.188 lr lrd reg : 0.05 0.7 0.4 Validation accuracy: 0.197 lr lrd reg : 0.05 0.75 0.2 Validation accuracy: 0.194 lr lrd reg : 0.05 0.75 0.25 Validation accuracy: 0.189 lr lrd reg : 0.05 0.75 0.3 Validation accuracy: 0.189 lr lrd reg : 0.05 0.75 0.35 Validation accuracy: 0.198 lr lrd reg : 0.05 0.75 0.4 Validation accuracy: 0.252 lr lrd reg : 0.1 0.3 0.2 Validation accuracy: 0.25 lr lrd reg : 0.1 0.3 0.25 Validation accuracy: 0.258 lr lrd reg : 0.1 0.3 0.3 Validation accuracy: 0.236 lr lrd reg : 0.1 0.3 0.35 Validation accuracy: 0.233 lr lrd reg : 0.1 0.3 0.4 Validation accuracy: 0.228 lr lrd reg : 0.1 0.35 0.2 Validation accuracy: 0.226 lr lrd reg : 0.1 0.35 0.25 Validation accuracy: 0.223 lr lrd reg : 0.1 0.35 0.3 Validation accuracy: 0.228 lr lrd reg : 0.1 0.35 0.35 Validation accuracy: 0.214 lr lrd reg : 0.1 0.35 0.4 Validation accuracy: 0.206 lr lrd reg : 0.1 0.4 0.2 Validation accuracy: 0.217 lr lrd reg : 0.1 0.4 0.25 Validation accuracy: 0.201 lr lrd reg : 0.1 0.4 0.3 Validation accuracy: 0.199 lr lrd reg : 0.1 0.4 0.35 Validation accuracy: 0.19 lr lrd reg : 0.1 0.4 0.4 Validation accuracy: 0.206 lr lrd reg : 0.1 0.45 0.2 Validation accuracy: 0.199 lr lrd reg : 0.1 0.45 0.25 Validation accuracy: 0.198 lr lrd reg : 0.1 0.45 0.3 Validation accuracy: 0.193 lr lrd reg : 0.1 0.45 0.35 Validation accuracy: 0.19 lr lrd reg : 0.1 0.45 0.4 Validation accuracy: 0.197 lr lrd reg : 0.1 0.5 0.2 Validation accuracy: 0.196 lr lrd reg : 0.1 0.5 0.25 Validation accuracy: 0.191 lr lrd reg : 0.1 0.5 0.3 Validation accuracy: 0.19 lr lrd reg : 0.1 0.5 0.35 Validation accuracy: 0.167 lr lrd reg : 0.1 0.5 0.4 Validation accuracy: 0.195 lr lrd reg : 0.1 0.55 0.2 Validation accuracy: 0.195 lr lrd reg : 0.1 0.55 0.25 Validation accuracy: 0.191 lr lrd reg : 0.1 0.55 0.3 Validation accuracy: 0.192 lr lrd reg : 0.1 0.55 0.35 Validation accuracy: 0.191 lr lrd reg : 0.1 0.55 0.4 Validation accuracy: 0.193 lr lrd reg : 0.1 0.6 0.2 Validation accuracy: 0.195 lr lrd reg : 0.1 0.6 0.25 Validation accuracy: 0.188 lr lrd reg : 0.1 0.6 0.3 Validation accuracy: 0.197 lr lrd reg : 0.1 0.6 0.35 Validation accuracy: 0.166 lr lrd reg : 0.1 0.6 0.4 Validation accuracy: 0.194 lr lrd reg : 0.1 0.65 0.2 Validation accuracy: 0.197 lr lrd reg : 0.1 0.65 0.25 Validation accuracy: 0.186 lr lrd reg : 0.1 0.65 0.3 Validation accuracy: 0.266 lr lrd reg : 0.1 0.65 0.35 Validation accuracy: 0.08 lr lrd reg : 0.1 0.65 0.4 Validation accuracy: 0.194 lr lrd reg : 0.1 0.7 0.2 Validation accuracy: 0.195 lr lrd reg : 0.1 0.7 0.25 Validation accuracy: 0.19 lr lrd reg : 0.1 0.7 0.3 Validation accuracy: 0.24 lr lrd reg : 0.1 0.7 0.35 Validation accuracy: 0.121 lr lrd reg : 0.1 0.7 0.4 Validation accuracy: 0.191 lr lrd reg : 0.1 0.75 0.2 Validation accuracy: 0.195 lr lrd reg : 0.1 0.75 0.25 Validation accuracy: 0.186 lr lrd reg : 0.1 0.75 0.3 Validation accuracy: 0.143 lr lrd reg : 0.1 0.75 0.35 Validation accuracy: 0.107 lr lrd reg : 0.1 0.75 0.4 best_val, set : 0.405 (0.01, 0.65, 0.25)

JinwoongKim / cs231n

assignment1-5 #5