JOSS tracking

[brentp]

re: https://github.com/openjournals/joss-reviews/issues/2119

grammar

• problems with sentence: This can be calculated intersecting (with bedtools,
• "regions between used defined" should be "... user-defined" ?

benchmarking

Thanks for the clear and reproducible benchmark scripts. (I have used this to reproduce your results and evaluate recent changes to mosdepth).

• It would be more fair to also show samtools depth without -a as that is easily converted to a BED format.
• either add whole-genome or a note suggesting why it's not there
• the disk vs streaming benchmarks don't seem that informative as they have the same run-times for the softwares; is there something to make the utility more apparent?

general

• samtools and mosdepth both handle CRAM files which are now a common format. It would be good to explain why this is not available in covtobed or add it as a known limitation.
• samtools and mosdepth both handle pair overlap and avoid double-counting overlapping mates. It would be good to clearly state how this is handled in covtobed.
• samtools and mosdepth both allow a mininum mapping quality and setting which flags are excluded/included and samtools allows a minimum base-quality. how is this handled in covtobed?
• please add more detail about the priority queue algorithm. does it consider cigar operations or does it append the entire read?

In summary, this tool seems to nicely fill a niche--streaming coverage analysis of target gene panels (when the data is in bam format and no knobs for mapping quality or flags, etc are needed). It would be nice to expand on how covtobed is unique to that niche (relative to samtools depth and an awk one-liner to convert to bed). Additional explanation of how this works: algorithm, filtering, etc will also be helpful.

[telatin]

Thank you for the prompt feedback, it's much appreciated! We will work to address all the questions

[brentp]

a couple more things:

I see now the mapping quality flag.

The automated tests don't seem to test the actual output, just the line counts. Would be nice to see some that check actual coverage.

Also, I think adding more obvious notes on strand, physical coverage etc. could make the specific, unique use-cases for this tool more apparent.

[telatin]

Thank you for the updates!

Before answering, I’d like to share a preamble:

• We decided to make an effort to keep the Wiki updated, and we would like to use it as the primary source of documentation (to avoid changing the repository to update the documentation, and to have an easy to browse document that will be probably more accessible than the paper itself, that we would like to keep concise). I expanded the Wiki with some of your suggestions
• We used the tool in different scenarios (microbial genomics, metagenomics, human genetics), and to be of general use we decided to do a little thing in the most flexible way (input/output streaming)

To address some points:

• PAPER: Thank you for the feedback on the paper: we updated it fixing the problematic bits and expanding it
• TESTS: There is a test that compares the produced BED with a reference BED in the repository. Today I added a second test that will check the actual output as suggested, in this case using a very artificial BAM file making the output easy to verify.
• CRAM: The limitation of not handling CRAM files has been added in the README and in the Wiki
• CIGAR: covtobed doesn’t parse the CIGAR and we do not plan to do so, as our goal is to check for reagions covered by the library, not covered by actual bases (i.e. evaluate the capture efficiency). We are going to clarify this, and this is why we compare the performance with “mosdepth -x”
• DOCUMENTATION: I added a second example of strand specific coverage usage in the wiki. The initial example was related to target enrichment using amplicon sequencing (to check for regions only covered by forward or by reverse reads), I added a recent usage on bacterial TraDIS.
• BENCHMARK: The whole-genome benchmark has been skipped for time constraints as we don’t have examples in house, but I’m now performing the test with a publicly available dataset. I removed samtools from the README as it can be misleading, and I added also “samtools depth” without the “-a” (only available in the subdirectory for the records).

[brentp]

Excellent. Thank you for explaining and updating.

There is a C++ htslib wrapper that is compatible with the bamtools API: https://github.com/38/HTSLibXX which would enable cram support. That's just a point of interest, not needed for the JOSS submission; you have addressed all my points.

[telatin]

Thank again for your contributions. I further expanded the wiki and the benchmark page with a first whole genome test and more details are about to be added.

We plan to try integrating HTSLibXX for a future update.