What should we summarize with interface output?

harvardinformatics / PhyloAcc-interface

Front-end for PhyloAcc

GNU General Public License v3.0

1 stars 1 forks source link

What should we summarize with interface output? #8

Open gwct opened 2 years ago

gwct commented 2 years ago

Building off of #7 and our recent discussions, I thought we could make a list here of possible ways to summarize the final output from PhyloAcc. A good starting point can be @xyz111131's original markdown document here: https://xyz111131.github.io/PhyloAcc/R/plot.html

Some of the things @tsackton and I have talked about already:

[x] Distributions of and correlations between Bayes Factors 1 and 2
[ ] Fraction of target branches accelerated vs. fraction of non-target branches accelerated per element
[ ] The difference between the Bayes Factor for the full model and the null model vs. the number of accelerated elements
[ ] Bayes factors vs. fraction of branches accelerated
[ ] A third Bayes factor: Full model vs. background model

Please leave other suggestions below and I will add them to this list!

xyz111131 commented 2 years ago

maybe: Bayes factors vs Fraction of branches accelerated. For branches accelerated, we might need to generate plots for different threshold of posterior.

gwct commented 2 years ago

Working on this now, but we've had discussions in the past about cutoffs for determining accelerated elements. I think for the ratites it was something like BF1 > 10 and BF2 > 1 and for the mammals it was BF1 > 5 and BF2 > 5. Is there any way we can determine these cutoffs for a given dataset, or at least provide guidance to the user as to what they should be?

tsackton commented 2 years ago

So, I am kind of inclined not to use Bayes Factor cutoffs by default. I think for some applications / research questions it is useful to use them, but if we generate a default cutoff then people will treat that as gospel and it will become a de facto P < 0.05. I think we should instead be encouraging a bit more of a Bayesian mindset. There are various rules of interpretation one can find by Googling we could include as a guide -- I am not sure if any of the actual statisticians have a preference.

xyz111131 commented 2 years ago

Yes I agree with Tim. It's hard to set a BF cutoff. Perhaps we can provide a minimum requirement, e.g. BF1 > 2, BF2 > 1?