benedekrozemberczki / SGCN

A PyTorch implementation of "Signed Graph Convolutional Network" (ICDM 2018).
GNU General Public License v3.0
268 stars 58 forks source link

How can I calculate AUPR in SGCN? #11

Closed songsong0425 closed 2 years ago

songsong0425 commented 2 years ago

Dear bendek, Greetings, I have a question about the calculation of AUPR in SGCN. I'm trying to get AUPR score by input the below code in utils.py:

from sklearn.metrics import roc_auc_score, f1_score, average_precision_score, precision_recall_curve, auc, plot_precision_recall_curve

def calculate_auc(targets, predictions, edges):
    """
    Calculate performance measures on test dataset.
    :param targets: Target vector to predict.
    :param predictions: Predictions vector.
    :param edges: Edges dictionary with number of edges etc.
    :return auc: AUC value.
    :return f1: F1-score.
    """
    targets = [0 if target == 1 else 1 for target in targets]
    auc_score = roc_auc_score(targets, predictions)
    #precision, recall, thresholds = precision_recall_curve(targets, predictions)
    #aupr=auc(recall, precision)
    pred = [1 if p > 0.5 else 0 for p in  predictions]
    f1 = f1_score(targets, pred)
    #precision, recall, thresholds = precision_recall_curve(targets, pred)
    #aupr=auc(recall, precision)
    pos_ratio = sum(pred)/len(pred)
    return auc_score, aupr, f1, pos_ratio

But I'm not sure where should I put the new code (#). Does AUPR need to get predictions value or pred value? Also, what the different between predictions and pred? I guess 'predictions' is probability value and 'pred' is binary value, is it right?

Sorry for frequent edit, but why did you set targets = [0 if target == 1 else 1 for target in targets]? I think that it means give opposite labels to positive and negative edges. (i.e., positive=0, negative=1) I look forward to your reply. Sincerely, Songyeon