search

EmmaRocheteau / TPC-LoS-prediction

This repository contains the code used for Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit (https://dl.acm.org/doi/10.1145/3450439.3451860).
https://dl.acm.org/doi/10.1145/3450439.3451860
MIT License
76 stars 25 forks source link

kappa for los #3

Closed XiangBu closed 3 years ago

XiangBu commented 3 years ago

hi, when I calculate kappa for los using sklearn from sklearn.metrics import cohen_kappa_score kappa = cohen_kappa_score(y_true, y_pred)

it occurred ValueError: continuous is not supported,it seems kappa is not proper for regression, only classification?

EmmaRocheteau commented 3 years ago

Hello! Yes the kappa score only works for categorical things. In fact we use a slightly modified kappa - the original kappa treats all categories as equal - i.e. getting the correct category is rewarded but incorrect categories are penalised equally. The linear weighted kappa ensures that categories that are close to the correct category e.g. 3-4 days instead of 2-3 days are "less bad" than predicting 8+ days. You can find more information about the linear weighted kappa score in this paper https://www.nature.com/articles/s41597-019-0103-9 as we used the same distribution bins as they did for ease of comparison. I hope that helps!

XiangBu commented 3 years ago

Hello! Yes the kappa score only works for categorical things. In fact we use a slightly modified kappa - the original kappa treats all categories as equal - i.e. getting the correct category is rewarded but incorrect categories are penalised equally. The linear weighted kappa ensures that categories that are close to the correct category e.g. 3-4 days instead of 2-3 days are "less bad" than predicting 8+ days. You can find more information about the linear weighted kappa score in this paper https://www.nature.com/articles/s41597-019-0103-9 as we used the same distribution bins as they did for ease of comparison. I hope that helps!

ok, got it. Thanks for your kind reply!