Improve run reports for rotary

[LeeBergstrand]

@jmtsuji Can you update the description for this with more explanation of what to do?

[jmtsuji]

Problem description

The run summary report currently generated by rotary is very basic. To understand what was done to each contig assembled by rotary and why (e.g., was the contig filtered by coverage, and why; how aggressively was it polished), a user currently needs to dig through several log files to find important information. It would be nice if we could generate more informative summary reports for rotary that collect this information for the user. Collecting this information for the user will also allow the user (and us, in end-to-end testing) to sanity-check the performance of rotary.

Proposed solution

I think the following types of reports could be helpful to create:

• Read QC report: summary of % reads retained vs. discarded during short and long read polishing
• Assembly report: the report already made by Flye could be used for this
• Contig stats report from assembly, polishing, and circularization: could include per-contig info on: contig length, whether it is circular, end repair status, changed bases during read polishing (if not too hard to add), coverage filtration status, and whether it was rotated to a start gene like dnaA
  • the current contig report generated by rotary (at "{sample}/stats/{sample}_contig_info.tsv") already kind of accomplishes this, but it only includes a simple Yes/No summary for each contig of whether it is circular, it was end-repaired successfully, and was retained during coverage filtration. More info could be added to this file to make a better contig report.
• Optional: Contig rotation report, to confirm that each contig was rotated to a new start position and then re-polished at least once during the pipeline. (It might actually be better to just confirm this by internal logic... a user might not need to see a report of this.)

After these reports are created, the Post-run tips section at the end of the README can be deleted, because this section summarizes which log files to manually check to obtain the above information.

Possible caveats

Because rotary has a lot of options that can be turned on or off (e.g., short read polishing; users can also just run up until the end of one module), we will need to consider how to make the reports modular so that the user can get reports at a variety of run endpoints. Rather than waiting to summarize all run info until the annotation module, we might need to add report summary rules within each module that summarize the information currently available. The next module can then take the run report from the previous module and add to it. I kind of do this already with "{sample}/stats/{sample}_contig_info.tsv"

[jmtsuji]

@jmtsuji Can you update the description for this with more explanation of what to do?

@LeeBergstrand Done! Thanks for breaking up the to-do file into individual issues.

[jmtsuji]

Quick update about the formats of the reports

To start, I was thinking that TSV files might be the most straightforward way to summarize the results from the different analysis steps. In general, one report could be made for each major bullet point above, with some exceptions (e.g., two reports might work better for read QC: one report for short reads and one report for long reads). In the longer term, we could consider making HTML reports with embedded tables and/or plots to summarize the rotary results (e.g., using info from the TSV files), if it's not too difficult to do this. I personally am happy with TSV files, but I wonder if users of the published version of the tool might appreciate a slightly more polished HTML report... @LeeBergstrand what are your thoughts?

[LeeBergstrand]

Quick update about the formats of the reports

To start, I was thinking that TSV files might be the most straightforward way to summarize the results from the different analysis steps. In general, one report could be made for each major bullet point above, with some exceptions (e.g., two reports might work better for read QC: one report for short reads and one report for long reads). In the longer term, we could consider making HTML reports with embedded tables and/or plots to summarize the rotary results (e.g., using info from the TSV files), if it's not too difficult to do this. I personally am happy with TSV files, but I wonder if users of the published version of the tool might appreciate a slightly more polished HTML report... @LeeBergstrand what are your thoughts?

@jmtsuji I want to use off-the-shelf tools to do much of the stats and make the HTML reports. For example, most of the reports ATLAS makes could have been generated by third-party tools and aggregated by MultiQC. This leads to less maintenance and its less likely that the reports will become broken.

[jmtsuji]

@LeeBergstrand Agreed - using off-the-shelf tools for both stats (as much as possible) and HTML reports sounds like a good idea for maintenance etc.. Relevant for #91

[LeeBergstrand]

QC Reports Generated Thus Far Per https://github.com/rotary-genomics/rotary/pull/214

• [x] Reads stats as generated by FASTQC
• [x] Reads removed before and after QC (short and long)
• [x] Assembly statistics a generated by QUAST
• [x] Aggregate statistics that show changes in assembly stats and qc across samples

[jmtsuji]

Thanks for these additions! We might also consider aggregating the coverage stats for each genome somehow. We might be able to just directly concatenate the coverage tables that are currently generated (via samtools coverage) for each sample.

[LeeBergstrand]

@jmtsuji That sounds good.

[LeeBergstrand]

Thanks for these additions! We might also consider aggregating the coverage stats for each genome somehow. We might be able to just directly concatenate the coverage tables that are currently generated (via samtools coverage) for each sample.

@jmtsuji This sounds like a good idea. Can you make a pull request for this?

[jmtsuji]

@LeeBergstrand PR is now made -- see #223