StatsDLMathsRecomSys / Inductive-representation-learning-on-temporal-graphs

297 stars 62 forks source link

something wrong with ap & roc in evaluation #15

Open zhaoyin214 opened 1 year ago

zhaoyin214 commented 1 year ago

ap & roc can not be calculated by mean()

            val_acc.append((pred_label == true_label).mean())
            val_ap.append(average_precision_score(true_label, pred_score))
            # val_f1.append(f1_score(true_label, pred_label))
            val_auc.append(roc_auc_score(true_label, pred_score))

https://github.com/StatsDLMathsRecomSys/Inductive-representation-learning-on-temporal-graphs/blob/66b2ccdd6d466f342900d8837c21224a49eda7e5/learn_edge.py#LL113C1-L116C66

it should be something like this


y_true = []
y_probs = []

for batch in dataloader:
    ...
    y_true.append(batch_y_true)
    y_probs.append(batch_y_probs)

...

ap = average_precision_score(y_true, y_probs)
roc = roc_auc_score(y_true, y_probs)