Closed XiangBu closed 3 years ago
Hello! Yes the kappa score only works for categorical things. In fact we use a slightly modified kappa - the original kappa treats all categories as equal - i.e. getting the correct category is rewarded but incorrect categories are penalised equally. The linear weighted kappa ensures that categories that are close to the correct category e.g. 3-4 days instead of 2-3 days are "less bad" than predicting 8+ days. You can find more information about the linear weighted kappa score in this paper https://www.nature.com/articles/s41597-019-0103-9 as we used the same distribution bins as they did for ease of comparison. I hope that helps!
Hello! Yes the kappa score only works for categorical things. In fact we use a slightly modified kappa - the original kappa treats all categories as equal - i.e. getting the correct category is rewarded but incorrect categories are penalised equally. The linear weighted kappa ensures that categories that are close to the correct category e.g. 3-4 days instead of 2-3 days are "less bad" than predicting 8+ days. You can find more information about the linear weighted kappa score in this paper https://www.nature.com/articles/s41597-019-0103-9 as we used the same distribution bins as they did for ease of comparison. I hope that helps!
ok, got it. Thanks for your kind reply!
hi, when I calculate kappa for los using sklearn from sklearn.metrics import cohen_kappa_score kappa = cohen_kappa_score(y_true, y_pred)
it occurred ValueError: continuous is not supported,it seems kappa is not proper for regression, only classification?