adyprat
commented
4 years ago Hi @Kaiton7,
The computeDGAUC.py considers edge direction by default (i.e., whether edge is from a -> b vs. b -> a). The effect of directed = True flag can be seen in lines 148-156 in https://github.com/Murali-group/Beeline/blob/master/BLEval/computeDGAUC.py. The length of outDF changes depending on whether the evaluation considers edge directions, i.e., outDF has (n*(n-1)) edges if directed = True, and (n choose 2) if directed = False, where n is the number of nodes in the given network.
When you mention that the networks our have both activation and repression edges, do you mean to ask if our AUPRC/AUROC evaluation additionally considers whether edge signs are predicted correctly? The current version of the AUC code does not consider edge signs, i.e., it only cares whether the direction of edge between two nodes a and b is predicted correctly (i.e., a -> b or b->a) and not whether the predicted edge a -> b is activating or inhibiting.
Note that the signed Early Precision is computed for activation and inhibition edges separately.
Hope that helps.
Kaiton7
Hello. Thank you for answering my question before.
Please teach me whether the program consider the edge's direction or not in calculating roc_curve . In /BLEval/computeDGAUC.py line221, I can see the only 1 or 0 in the dataframe (outDF["TrueEDges"]). but in the paper(https://www.biorxiv.org/content/10.1101/642926v3.full#ref-10) about the Synthetic data it shows that there are activation and repression edges. I think there are algorithms that consider the edge's direction and not, that's why they don't consider the direction in evaluation. I'm sorry if I'm saying too directly or impolitely. I appreciate for you and this program.