replace sigmoid with relu, but useless

[yunshuipiao]

Hello. Thank you for code. I want to replace sigmoid with with relu, but cant not improve the accuracy_score。 modify code：

def sigmoid(z):
#     return 1.0 / (1.0 + np.exp(-z))
    return z * (z > 0)
def sigmoid_prime(z):
#     return sigmoid(z) * (1 - sigmoid(z))
    return 1.0 * (z > 0)

but ：

Epoch 0: 401 / 4200, Accuracy score: 0.09547619047619048
Epoch 1: 401 / 4200, Accuracy score: 0.09547619047619048
Epoch 2: 401 / 4200, Accuracy score: 0.09547619047619048

Thanks for help me.

[schirmacher]

Use a learning rate of 0.01 and initialize weights and biases to a much smaller value ( * 0.1). Epoch 29 : 9677 / 10000