loss - Githubissues

ih4cku / blog

deprecated, Git issues are great for writing blogs :)

2 stars 0 forks source link

loss #41

Open ih4cku opened 8 years ago

ih4cku commented 8 years ago

multinomial logistic loss
cross entropy loss
softmax loss

ih4cku commented 8 years ago

http://stats.stackexchange.com/questions/166958/multinomial-logistic-loss-vs-cross-entropy-vs-square-error

ih4cku commented 8 years ago

Loss function is to measure the quality of model output.

The typical loss function for classification problem is:

ih4cku commented 8 years ago

Logistic function: $f(x)=\frac{L}{1+e^{-k(x-x_0)}}$

Sigmoid function is a special case of the Logistic function when $L=1, k=1, x_0=0$ : $\sigma(x)=\frac{1}{1-e^{-x}}$

The output range of the sigmoid function is [0, 1].

The first derivative of sigmoid function is: $\frac{\partial\sigma}{\partial x}=\sigma(x)(1-\sigma(x))$

ih4cku commented 8 years ago

In the linear classifier: $f(x;W,b)=Wx+b$ , each row of W is a classifier. Each row of W can corresponds to a template of the category. So, the classifier is doing template matching.