Difficulty in understanding xent function

[utkarsh0902311047]

Hi, Thank you so much for the detailed code. I would highly appreciate your explaining the xent function with np.arange and np.argmax. What is it actually doing?

[zjost]

Hello. You mean this?

def xent(pred, labels):
    return -np.log(pred)[np.arange(pred.shape[0]), np.argmax(labels, axis=1)]

Recall that cross entropy is $- \sum_c y_c \log p_c$, where $y_c$ is 1 if c is the true class, and zero otherwise. Likewise, $p_c$ is the prediction that the class is $c$. The term in the sum is zero except for the one case where $y_c = 1$, which is given by np.argmax(labels, axis=1). So we calculate the negative log likelihood, and then select the one entry where $y_c=1$. I don't know why I'm using np.arange here, because this is just selecting everything and could have just used :.