Murali-group / Beeline

BEELINE: evaluation of algorithms for gene regulatory network inference
GNU General Public License v3.0
171 stars 53 forks source link

SCNS ranked edges #61

Closed TiamHeydari closed 2 years ago

TiamHeydari commented 2 years ago

Hi, I am trying to understand the benchmarking of SCNS method. As I understand SCNS gives a single GRN as the fit. How you compare it with other methods? Thanks

adyprat commented 2 years ago

Hi,

SCNS does not output a single GRN, rather it outputs potential regulatory rules for each gene in a separate text file. Sometimes it predicts over a 100+ possible rules for each gene. These predicted rules may not even have any overlap in the regulators and can even be conflicting in some cases.

For example, if the network comprises of four nodes, A, B, C, and D. Here's what a sample output from SCNS looks like:

Predicted rules for A in A.txt (B or C) (B or C) and not D (B and D)

Predicted rules for B in B.txt (A or C) (A or D) and (A or C)

and so on...

We then chose the following approach to convert this into a ranked list of edges to compare it with other methods:

For each gene, we count the number of times its regulator appears in all possible predicted rules and use that number as a measure of confidence that the regulator has some effect on the gene. So, the final rankedEdges.txt looks like:

Gene1(from),Gene2(to),Score B,A 3 (since it appears three times in A.txt, three rules) C,A,2 D,A,2 A,B,3 (A appears three times in B.txt, even though it appears only in two rules) C,B,2 D,B,1 ...

While there are possibly other ways we could have parsed these rules, this approach seemed reasonable at the time. Hope this helps.

Best, Aditya

TiamHeydari commented 2 years ago

Hi Aditya, Thanks very much for the explanation, it makes sense. Best, Tiam