The issue with method #1 is that you can't implement the learning rate portion that you wanted us to use, so I am assuming it's method #2 but I wanted to clarify with you. Please let me know.
Use method 1 (Keras). We haven't talked about backprop in detail.
You'll do method 2 on the next homework.
You can set the learning rate in the optimizer.
Hey Jordan, looking at problem 2.4, how do you want us to implement the neural network? Do you want us to use:
Method #1 model = Sequential() model.add(Dense(256, activation='relu',input_shape=(784,))) model.compile(loss=keras.losses.categorical_crossentropy,optimizer=keras.optimizers.Adadelta(),metrics=['accuracy']) model.fit(x_train, y_train,batch_size=32,epochs=10,verbose=1,validation_data=(x_test, y_test))
Method #2 alpha = 0.01 # set learning rate theta_1 = np.random.normal(0,.1,size=(2,3)); b1 = np.zeros((1,3)) # init weights theta_2 = np.random.normal(0,.1,size=(3,2)); b2 = np.zeros((1,2))
J = []
for i in range(10000): l1 = relu(np.dot(X, theta_1) + b1) # l1 = X theta_1 y_hat = softmax(np.dot(l1, theta_2) + b2) # Y_hat = l1 theta_2 + b
The issue with method #1 is that you can't implement the learning rate portion that you wanted us to use, so I am assuming it's method #2 but I wanted to clarify with you. Please let me know.