search

markziemann / enrichment_recipe

A recipe for extremely reproducible enrichment analysis
MIT License
2 stars 1 forks source link

Comments on enrichment letter #32

Closed anusuiyaxbora closed 10 months ago

anusuiyaxbora commented 10 months ago

Some edits and comments in between the text (written in bold and italics):

  1. They also require little training to use, which on one hand simplifies bioinformatics tasks, but on the other hand these tools can easily be misused and results misreported, which can lead to misleading conclusions [3,4]. Researchers also have bias towards tools that are commonly used and do not conduct their own due diligence on reliability of the results obtained by various web-based tools; there is negligence on verifying the results as it can be a time consuming process. Last year we sought to test whether a small group of articles published in 2019 involving enrichment analysis with the DAVID tool [5] were reproducible using the authors’ own methods. We found only 4/20 enrichment studies had a high degree of reproduction, while 7/20 had severe discrepancies. After our pilot study completed, we were surprised to hear that the most commonly used version of DAVID till 2022 for all of these studies (v6.8) would no longer be available from June 2022 onwards.

  2. This is a prime example of “link decay,” a phenomenon where internet-based resources are lost over time, which has been raised as a significant and ongoing problem for bioinformatic reproducibility [6]. Can add About articles using enrichment tools just for the sake of it as it is evident by exclusion of DAVID results from conclusions of the paper? The problem of non-reproducibility is alarming as the presence of multiple articles using DAVID could be potentially used for pre-clinical studies.

  3. Most of the articles are not concerned to mention versions used in their analysis... it is up to the interpretation of the reader. There should be guidelines by the journals to enforce minimum criteria to mention web-tools and softwares in the papers? Researchers should keep in mind that institutional and funding mandates for data retention also apply to software and algorithms [7,8].

Will edit more if you like these points...

markziemann commented 10 months ago

Thanks so much Anusuiya! I have made some changes based on your comments and I hope you can include these in your thesis as well.

anusuiyaxbora commented 10 months ago

@markziemann Hi Mark, how about the alt title that you thought of before - "Time to abandon (most) web-based tools in genomics"

Sounds crisp and might gather more interest...

markziemann commented 10 months ago
  1. Good point - I would like to include a mention of researchers using outdated tools https://www.the-scientist.com/news-opinion/scientists-continue-to-use-outdated-methods-30438 but I don't see any easy place for us to put it.
  2. The point is a bit inconsistent. For the sake of simplicity for this paper we should say that enrichment analysis is important for the conclusions of the article.
  3. Yes we should mention the reporting checklist and role of reviewers and journals in the concluding remarks