Time to reboot BACTpipe (version 3)

[boulund]

I think it is time to review the BACTpipe workflow and update/replace some parts.

Looking for input @huyue87 and @thorellk!

I have the following ideas:

• Remove mash screen (#54). It was originally intended as a screening step to not unnecessarily assemble potentially contaminated isolates, but in reality we assemble all samples anyway so this just wastes resources (time, CPU, memory) when it can easily be replaced by a sendsketch.sh run on the assembled contigs. If the sample is contaminated we will still see it in the assembled contigs.
• Replace BBDuk with fastp. Fastp has the benefit of producing quality statistics plots out-of-the-box, so it removes the need for a separate FastQC run. And fastp output can easily be summarized with MultiQC (support already exists). It is about 10x slower than BBDuk in my testing, but is slightly more accurate and the time gained by removing mash screen and FastQC should make up for it I think.
• License issue and SignalP-dependency of Prokka. I want to make BACTpipe more portable and reproducible, since we've been having some issues with reproducibility even on our own server. I want to put all dependencies in a single Docker container (when we've decided which versions of all tools should be included). Then we can install the academic version of SignalP in the container, which should be fine license-wise as long as we don't make the docker image public. We could still make a public version where people can link in their local installations of SignalP if we want to (it's fairly easy to tell docker to mount a local directory inside a container on runtime).
• Update all main tools to the most recent versions (#104):
  • BBMap 38.63
  • Shovill 1.0.4
  • Prokka 1.14.0
  • MultiQC 1.7
  • Fastp 0.20.0
  • SignalP 5.0b

Please share any comments or ideas on how we can streamline and improve BACTpipe. I remember we've also talked about potentially including some MLST tools, genome assembly assessment tools (e.g. #103), or phage finding tools (#96).

[thorellk]

Replace mash screen with sendsketch.sh - Sounds good.

Replace BBDuk with fastp. - I never used fastp but your arguments sounds reasonable -> approved :)

License issue and SignalP-dependency of Prokka. - I think this sounds reasonable but have no experience in making Docker container myself so I don't know how much work this would be?

Update all main tools to the most recent versions. - No brainer

Assembly assessment tools I think could be nice. Bacteria are not mentioned at all in BUSCO, it seems to describe application to larger genomes. So we should just double check that this is proper, or look into alternatives.

I think we should stick to these updates first and then add more things if/when there is time + an actual, concrete and general (in most projects) need for it.

[boulund]

@huyue87 and I brainstormed a bit about potential improvements to BACTpipe in version 3. We considered rewriting BACTpipe to something like the following workflow graph.

In this suggestion we use:

• FastP for FASTQ quality and adapter trimming
• Shovill for assembly
• BBMap sendsketch.sh for quick taxonomic screening/classification
• BacMet for metal resistance
• Prokka for general annotation. We could also consider to use DFAST here instead
• VirulenceFinder for virulence profiling
• MLST for sequence typing
• [PlasmidFinder]() for plasmid finding
  • pMLST for plasmid typing

The suggested outputs will be:

• FastP quality reports (HTML)
• Entire Prokka output (or DFAST)
• Individual output files from each tool
• A summary table that merges all information from VirFinder, MLST, Plasmidfinder+pMLST, and BacMet into a joint table for easy overview.

Open questions

• Do we need a MultiQC summary of the FastP reports?
• Is it possible to cleanly integrate hybrid assemblies (e.g. Nanopore + short reads) in the suggested workflow?
• Add more extensive assembly quality assessment? We previously ran BBMap's stats.sh, but maybe we want to run Quast? We also previously discussed BUSCO. Another alternative could be e.g. CheckM, but I have never used that for isolate assemblies.

[boulund]

@thorellk @huyue87 and I recently had a meeting to plan for the work ahead. We decided to also add ResFinder to the suite of tools to be run. BACTpipe is really becoming a CGE workflow :laughing: ...

We also decided to split up development efforts into version 3 and 4. Version 3 will focus on getting the basic short read workflow in place, and version 4 will add support for long reads. We committed a few draft workflow graphs to docs/source/img on the develop branch.

[valzip]

Hi! I found your pipeline while looking for some program that would screen assembled contigs for contamination in my microbial pipeline. I am glad that my pipeline is looking roughly similar to yours. I can recommend you abricate from tseemann for virulence and resistance finding. It implements several databases, it is easy to run, runs fast, and in silico results are in agreement with our wet lab results.

[boulund]

Hi @valzip ! Thanks for sharing! We also love pretty much everything Torsten does :D. We've actually discussed abricate previously, and it's not at all impossible that we might decide to use abricate instead in the end.