Problem with prediction code / size of the weight vector

[vchahun]

This is an interesting project! However, the code seems to be doing different things from what you describe in your blog post.

1. Shouldn't the sentence "We will then assign the class j if the prediction wTX lies in the interval [θj−1,θj[" translate into:

def ordinal_logistic_predict(w, theta, X):
    """
    Parameters
    ----------
    w : coefficients obtained by ordinal_logistic
    theta : thresholds
    """
    unique_theta = np.sort(np.unique(theta))
    out = X.dot(w)
    unique_theta[-1] = np.inf # p(y <= max_level) = 1
    tmp = out[:, None].repeat(unique_theta.size, axis=1)
    return np.argmax(tmp < unique_theta, axis=1)

If I make this change, I obtain much better performance, but it seems almost too perfect:

MEAN ABSOLUTE ERROR (ORDINAL LOGISTIC):    2.88859180036
MEAN ABSOLUTE ERROR (LOGISTIC REGRESSION): 3.83957219251
MEAN ABSOLUTE ERROR (RIDGE REGRESSION):    3.5623885918

1. Why is the threshold vector of the same size as the number of levels K? I believe it should have size K-1, but I am not sure of how to modify the gradient to have this...

[fabianp]

hi @vchahun !

You are right that my function ordinal_logistic_predict is a bit different from what is said in my blog. In the code instead of using K+1 thresholds I'm using the K means between the different thresholds, that's why there is K levels instead of K+1.

For me, the scores are almost identical (3.831 vs 3.839). I have changed it nevertheless to your implementation since it seems more consistent with the cost function we are using.

It should be fixed in 7c655840fa67ab391ca7f5576ddd9d2af33c25cf . Thanks!