[PRE REVIEW]: CRBHits: Conditional Reciprocal Best Hits in R

[whedon]

Submitting author: @kullrich (Kristian Ullrich) Repository: https://gitlab.gwdg.de/mpievolbio-it/crbhits Version: v0.0.1 Editor: @KristinaRiemer Reviewers: @clauswilke, @a-r-j Managing EiC: Daniel S. Katz

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @kullrich. Currently, there isn't an JOSS editor assigned to your paper.

@kullrich if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.14 s (625.3 files/s, 85785.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                            34           1540            385           6248
R                               36             59            562           1008
CSS                              3             99             48            428
Markdown                         2             98              0            268
JavaScript                       3             64             32            256
TeX                              1             13              0            132
Rmd                              3            172            339            104
YAML                             4              3              0             66
SVG                              1              0              1             11
-------------------------------------------------------------------------------
SUM:                            87           2048           1367           8521
-------------------------------------------------------------------------------

Statistical information for the repository '2258' was gathered on 2020/05/26.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
kristian.ullrich                 3           704            352          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
kristian.ullrich            352           50.0          0.0                9.09

[whedon]

Reference check summary:

OK DOIs

- 10.1371/journal.pgen.1004365 is OK
- doi:10.21105/joss.00142 is OK
- 10.1101/gr.113985.110 is OK
- 10.1186/s13059-015-0721-2 is OK
- 10.1093/protein/12.2.85 is OK
- 10.1007/978-3-540-35306-5_10 is OK
- 10.1016/S1672-0229(10)60008-3 is OK
- 10.1007/BF02407308 is OK
- 10.18129/B9.bioc.Biostrings is OK
- 10.1093/oxfordjournals.molbev.a026236 is OK
- 10.1016/j.cbpa.2003.12.004 is OK
- 10.1093/nar/gkr853 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[danielskatz]

👋 @kullrich - Thanks for your submission to JOSS. As described in our blog post announcing the reopening of JOSS, we're currently working in a "reduced service mode", limiting the number of papers assigned to any individual editor.

Since reopening JOSS last week, we've had > 60 papers submitted and as such, yours has been put in our backlog that we will be working through over the coming weeks and months.

Thanks in advance for your patience!

(Also, I see some small issues in the paper, such as the authors for the R reference, and some awkward grammar in the text - feel free to make change to the bib or md files, and enter @whedon generate pdf as a new comment here to check the results)

[kullrich]

Hi,

I will work on it and generate the paper with the @whedon generate pdf asap.

Best regards

Kristian

Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 26. May 2020, at 19:13, whedon notifications@github.com wrote:

👉 Check article proof 📄 👈 https://github.com/openjournals/joss-papers/blob/joss.02258/joss.02258/10.21105.joss.02258.pdf — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2258#issuecomment-634157971, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5ZWIY622GYBZI2XVG2BIDRTP2DNANCNFSM4NKXBB3A.

[kullrich]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kullrich]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kullrich]

Possible reviewer based on their preferred program language R and their topic area, starting from the bottom of the list of people agreed to review: davismcc richelbilderbeek druvus dcassol markziemann TGuillerme vbaliga clauswilke

[kullrich]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kullrich]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[labarba]

hi @kullrich -- I am the Associate Editor-in-Chief on rotation this week. I'm going to send this back to you for some more work, because the paper does not meet our expectations. In particular, see the section of our Documentation about the Software paper. We need:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? State of the field: Do the authors describe how this software compares to other commonly-used packages?

As a non-specialist, your summary is quite a slog to get even through the first paragraph. It would help us quite a bit to identify handling editor and reviewers if you tell us in plain words what the software does.

[kullrich]

Dear Lorena A. Barba,

I will try to make this more clear in the summary.

However, I wonder since the CRBH algorithm and its python implementation was published in JOSS, however the python implementation is not functional at the moment, which is one reason why I ported the whole pipeline into R introducing additional features and downstream functionality.

Here is the original paper: https://joss.theoj.org/papers/10.21105/joss.00142

Given this, the summaries of both papers share the need for the implementation of CRBH.

See Scott 2017:

"Conditional Reciprocal Best Hits (CRBH) was originally described by Aubry et al.(Aubryet al. 2014) and implemented in the crb-blast package. CRBH is a method for finding orthologs between two sets of sequences which builds on the traditional Reciprocal BestHits (RBH) method;”

See this manuscript:

"As described earlier (Aubry et al., 2014; Scott, 2017), CRBH builds upon the classical reciprocal best hit (RBH) approach to find orthologous sequences between two sets of sequences by defining an expect-value cutoff per alignment length.”

I wonder, why the summary of Scott 2017 was sufficient for a broader audience and it is not for this manuscript.

I try to make this more clear.

Do I need to resubmit as a new submission or can I work on the article alone?

Best regards

Kristian Ullrich

-- Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 12. Jun 2020, at 19:51, Lorena A. Barba notifications@github.com wrote:

hi @kullrich https://github.com/kullrich -- I am the Associate Editor-in-Chief on rotation this week. I'm going to send this back to you for some more work, because the paper does not meet our expectations. In particular, see the section of our Documentation about the Software paper https://joss.readthedocs.io/en/latest/review_checklist.html?highlight=summary#software-paper. We need:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? State of the field: Do the authors describe how this software compares to other commonly-used packages?

As a non-specialist, your summary is quite a slog to get even through the first paragraph. It would help us quite a bit to identify handling editor and reviewers if you tell us in plain words what the software does.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2258#issuecomment-643406755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5ZWI2V3JWFPH6ZFC5V6XTRWJTINANCNFSM4NKXBB3A.

[labarba]

You can work on your paper in-place. No need for a new submission.

Like in every journal, peer-review is done by humans, and both editors and reviewers interpret the journal's policies somewhat differently. As the Associate Editor-in-Chief who processed this incoming submission, all I can say is that your paper is hard to understand for a non-specialist like me, and thus it's hard for me to match it to a handling editor.

[kullrich]

Dear Lorena A. Barba,

thank you for your quick response.

It is good to know that I can directly work on the paper without resubmitting.

I hope that I can make it more clear and will extend the summary and audience part.

The software is a bioinformatic tool for population and evolutionary biologist, comparing nucleotide sequences from different species or from the same species to each other (orthologous sequences).

I also went through your reviewer list and indicated already some in my opinion valid candidates who have been working on either evolutionary or population genetic related software development and research projects.

I think Claus Wilke would be a perfect reviewer, however he was actual more at the lower part of your possible reviewer list.

Thank you again for handling this manuscript.

Best regards

Kristian Ullrich

davismcc richelbilderbeek druvus dcassol markziemann TGuillerme vbaliga clauswilke

Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 13. Jun 2020, at 16:06, Kristian Ullrich ullrich@evolbio.mpg.de wrote:

Dear Lorena A. Barba,

I will try to make this more clear in the summary.

However, I wonder since the CRBH algorithm and its python implementation was published in JOSS, however the python implementation is not functional at the moment, which is one reason why I ported the whole pipeline into R introducing additional features and downstream functionality.

Here is the original paper: https://joss.theoj.org/papers/10.21105/joss.00142 https://joss.theoj.org/papers/10.21105/joss.00142

Given this, the summaries of both papers share the need for the implementation of CRBH.

See Scott 2017:

"Conditional Reciprocal Best Hits (CRBH) was originally described by Aubry et al.(Aubryet al. 2014) and implemented in the crb-blast package. CRBH is a method for finding orthologs between two sets of sequences which builds on the traditional Reciprocal BestHits (RBH) method;”

See this manuscript:

"As described earlier (Aubry et al., 2014; Scott, 2017), CRBH builds upon the classical reciprocal best hit (RBH) approach to find orthologous sequences between two sets of sequences by defining an expect-value cutoff per alignment length.”

I wonder, why the summary of Scott 2017 was sufficient for a broader audience and it is not for this manuscript.

I try to make this more clear.

Do I need to resubmit as a new submission or can I work on the article alone?

Best regards

Kristian Ullrich

-- Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de mailto:ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 12. Jun 2020, at 19:51, Lorena A. Barba <notifications@github.com mailto:notifications@github.com> wrote:

hi @kullrich https://github.com/kullrich -- I am the Associate Editor-in-Chief on rotation this week. I'm going to send this back to you for some more work, because the paper does not meet our expectations. In particular, see the section of our Documentation about the Software paper https://joss.readthedocs.io/en/latest/review_checklist.html?highlight=summary#software-paper. We need:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? State of the field: Do the authors describe how this software compares to other commonly-used packages?

As a non-specialist, your summary is quite a slog to get even through the first paragraph. It would help us quite a bit to identify handling editor and reviewers if you tell us in plain words what the software does.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2258#issuecomment-643406755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5ZWI2V3JWFPH6ZFC5V6XTRWJTINANCNFSM4NKXBB3A.

[kullrich]

@whedon https://github.com/whedon generate pdf

Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 13. Jun 2020, at 17:03, Kristian Ullrich ullrich@evolbio.mpg.de wrote:

Dear Lorena A. Barba,

thank you for your quick response.

It is good to know that I can directly work on the paper without resubmitting.

I hope that I can make it more clear and will extend the summary and audience part.

The software is a bioinformatic tool for population and evolutionary biologist, comparing nucleotide sequences from different species or from the same species to each other (orthologous sequences).

I also went through your reviewer list and indicated already some in my opinion valid candidates who have been working on either evolutionary or population genetic related software development and research projects.

I think Claus Wilke would be a perfect reviewer, however he was actual more at the lower part of your possible reviewer list.

Thank you again for handling this manuscript.

Best regards

Kristian Ullrich

davismcc richelbilderbeek druvus dcassol markziemann TGuillerme vbaliga clauswilke

Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de mailto:ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 13. Jun 2020, at 16:06, Kristian Ullrich <ullrich@evolbio.mpg.de mailto:ullrich@evolbio.mpg.de> wrote:

Dear Lorena A. Barba,

I will try to make this more clear in the summary.

However, I wonder since the CRBH algorithm and its python implementation was published in JOSS, however the python implementation is not functional at the moment, which is one reason why I ported the whole pipeline into R introducing additional features and downstream functionality.

Here is the original paper: https://joss.theoj.org/papers/10.21105/joss.00142 https://joss.theoj.org/papers/10.21105/joss.00142

Given this, the summaries of both papers share the need for the implementation of CRBH.

See Scott 2017:

"Conditional Reciprocal Best Hits (CRBH) was originally described by Aubry et al.(Aubryet al. 2014) and implemented in the crb-blast package. CRBH is a method for finding orthologs between two sets of sequences which builds on the traditional Reciprocal BestHits (RBH) method;”

See this manuscript:

"As described earlier (Aubry et al., 2014; Scott, 2017), CRBH builds upon the classical reciprocal best hit (RBH) approach to find orthologous sequences between two sets of sequences by defining an expect-value cutoff per alignment length.”

I wonder, why the summary of Scott 2017 was sufficient for a broader audience and it is not for this manuscript.

I try to make this more clear.

Do I need to resubmit as a new submission or can I work on the article alone?

Best regards

Kristian Ullrich

-- Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de mailto:ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 12. Jun 2020, at 19:51, Lorena A. Barba <notifications@github.com mailto:notifications@github.com> wrote:

hi @kullrich https://github.com/kullrich -- I am the Associate Editor-in-Chief on rotation this week. I'm going to send this back to you for some more work, because the paper does not meet our expectations. In particular, see the section of our Documentation about the Software paper https://joss.readthedocs.io/en/latest/review_checklist.html?highlight=summary#software-paper. We need:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? State of the field: Do the authors describe how this software compares to other commonly-used packages?

As a non-specialist, your summary is quite a slog to get even through the first paragraph. It would help us quite a bit to identify handling editor and reviewers if you tell us in plain words what the software does.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2258#issuecomment-643406755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5ZWI2V3JWFPH6ZFC5V6XTRWJTINANCNFSM4NKXBB3A.

[whedon]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

[kullrich]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kullrich]

Dear Lorena A Barba,

I worked on the paper and hope that it now meet your journal expectations.

Summary:

I added a more clear description what Conditional Reciprocal Best Hit approach is about and that it builds on a common task in comparative genomics, namely RBH and ortholog detection.

"The Reciprocal Best Hit (RBH) approach is commonly used in bioinformatics to show that two sequences evolved from a common ancestral gene. In other words, RBH tries to find orthologous protein sequences within and between species. These orthologous sequences can be further analysed to evaluate protein family evolution, infer phylogenetic trees and to annotate protein function (Altenhoff, Glover, & Dessimoz, 2019).”

I also added a more clear description that CRBH tries to add additional orthologs as compared to the traditional RBH approach.

“CRBH uses the sequence search results to fit an expect-value (e-value) cutoff given each RBH to subsequently add sequence pairs to the list of bona-fide orthologs given their alignment length.”

For the statement of need, I highlighted in the second sentence a more clear description of the target community, namely ecology, population and evolutionary biologists. Which among other scientific communities use the R environment for their research. I have not listed the amount of R packages related to population, ecology and evolution, however I guess it might be a lot. The Biostrings Bioconductor package e.g. has a download statistic of more than 200k per year since 2014.

For the state of the field: A direct comparison for the primary intention of the R package, the reimplementation of the CRBH approach in the R environment is now given in Table 1 highlighting the Runtime of both the CRBHits R implementation and the python implementation of CRBH of shmlast published in JOSS in 2017.

Best regards

Kristian Ullrich

-- Kristian Ullrich, Ph.D. Scientific IT group Department of Evolutionary Genetics Max Planck Institute for Evolutionary Biology August Thienemann Str. 2 24306 Plön Germany +49 4522 763 313 ullrich@evolbio.mpg.de

-- CONFIDENTIALITY NOTICE: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.

On 13. Jun 2020, at 22:41, whedon notifications@github.com wrote:

👉 Check article proof 📄 👈 https://github.com/openjournals/joss-papers/blob/joss.02258/joss.02258/10.21105.joss.02258.pdf — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2258#issuecomment-643675626, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5ZWI7K4OJW75W2ZADNR33RWPP67ANCNFSM4NKXBB3A.

[Kevin-Mattheus-Moerman]

@labarba :point_up: can you follow up on these changes? Thanks

[Kevin-Mattheus-Moerman]

@KristinaRiemer @usethedata @marcosvital @will-rowe @lpantano could one of you handle this submission?

[KristinaRiemer]

I can take this submission if @labarba thinks it's ready.

[labarba]

I'm going to leave the assessment of the paper to the handling editor and reviewers, moving forward.

[KristinaRiemer]

@whedon assign me as editor

[whedon]

OK, the editor is @KristinaRiemer

[kullrich]

Travis CI fails at the moment for R-devel, R-release compiles without any warnings. I have already opene an issue on Bioconductor: https://support.bioconductor.org/p/131811/ regarding this issue. Should I remove R-devel from travis.yml for the time of review?

[kullrich]

Travis CI now builds also for R-devel without any errors. Needed to change the travis.yml as indicated here https://support.bioconductor.org/p/131811/.

[KristinaRiemer]

Thanks for providing a list of potential reviewers @kullrich. @davismcc @druvus @vbaliga would any of you be able to review this submission?

[vbaliga]

Hi @KristinaRiemer, I am generally available to review submissions but after taking a look through this one, I feel that the content is a little too far outside my domain of expertise for me to be able to adequately assess it. Thanks for the opportunity, but I think you may be better off with someone else

[KristinaRiemer]

@clauswilke @dcassol @davismcc @druvus would any of y'all be able and willing to review this submission?

[clauswilke]

I can review this.

[KristinaRiemer]

Great, thanks so much @clauswilke!

[KristinaRiemer]

@whedon assign @clauswilke as reviewer

[whedon]

OK, @clauswilke is now a reviewer

[dcassol]

@KristinaRiemer Unfortunately, I am not available this time.

[KristinaRiemer]

I'm still looking for one more reviewer for this submission. @richelbilderbeek @TGuillerme @sk-sahu @a-r-j @jspaezp would any of you be available and willing to review this for JOSS?

[a-r-j]

I can review this.

[TGuillerme]

@KristinaRiemer Sorry I can't review it this time, I am already over committed for other community services.

[sk-sahu]

Good to know @a-r-j already agreed. But If more hands required please let me know.

[KristinaRiemer]

Great, thanks so much @a-r-j! And thank you for offering up help @sk-sahu, I'll definitely let you know if we need it.

[KristinaRiemer]

@whedon add @a-r-j as reviewer

[whedon]

OK, @a-r-j is now a reviewer

[KristinaRiemer]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/2424.

[KristinaRiemer]

Okay @clauswilke @a-r-j, the review will happen in issue #2424. Let me know over there if you have any questions!