VirSorter filtering

[hoelzer]

I saw that you filter the VirSorter output to only collect phages from the files

• VIRSorter_cat-1.fasta
• VIRSorter_cat-2.fasta

cat !{results}/Predicted_viral_sequences/VIRSorter_cat-[1,2].fasta | grep ">" | sed -e s/\\>VIRSorter_//g | sed -e s/-cat_1//g |\
  sed -e s/-cat_2//g  > virsorter.txt

Why not also include the cat-3 phages? To reduce the amount of false-positive hits? However, to have a somewhat fair comparison with the other tools it might be worth to also include cat-3 phages identified by VirSorter?

And prophages you are not interested at all?

[replikation]

@Stormrider935 please read up on what to include ;)

[replikation]

The current status is (i think) that filtering is still work in progress, between these tools. So if you may have additional insights in what to include, feel free to add ;)

[mult1fractal]

yes i will check later.. the first version of all filter scripst were included to generate some output for replikation's heatmap.

[mult1fractal]

• here is my suggestion for the filter scripts

• the goal of our filter scripts was to reduce the output of the phage tools because otherwise the heatmap was too large

• my idea was that " when i search for phages i need to know what is in my sample until a certain possibility"

• filter script --> Output: hit and possible hit (in the table)

• some filter scripts meet the requirements in the table, some not, maybe you(@hoelzer) can check if it make sense what i've written down (if yes i would implement it like this)

• @replikation I think it would make sense if we tell the users in the readme about the result filter so they know the result they get is only part of the whole output of all tools

overview phage tools parameters

Tool	criterion	hit	possible hit	meh
MARVEL	%	100-75	74.9 -50	49.9-0
VirFinder	score (the higher the score, the higher hte possibilityfor a phage hit) A score of 1 represents perfect identification of all true viral contigs with no false positives, and ascore of 0.5 represents a random classification	0.999-0.75	0.749 - 0.5	0.499 - 0.0
PPR-Meta	phage	phage
VirSorter	sure and somwhat sure (filters for prophage and complete phage contig)	sure pvalue under 0.01	somewhat sure	not so sure
MetaPhinder	classification : Phage	Phage	-	negative
DeepVirFinder	the higher the score, the higher the possibility for a phage hit	0.999-0.75	0.749 - 0.5	0.499 - 0.0

include hit and possible hit

MARVEL

quote from paper

A contig was considered viral if predicted in categories I and II for VirSorter, and if the q-value was less than or equal to 0.01 for VirFinder

• yes no result

Virfinder

quote from paper

A score of 1 represents perfect identificationof all true viral contigs with no false positives, and a score of 0.5 represents a random classification.

• p-value 0-1 gradation of result (other result display)

PPR-Meta

quote from paper

difference between the phage score and chromosome score reveals the lifestyle of the phages (virulent or temperate), while the difference between the plasmid score and chromosome score reveals the transmissibility of the plasmids (transmissible or non-transmissible). the category with the highest score is selected as our prediction. unsicher ob phage score dem pvalue entspricht

• phage score 0-1 gradation of result (other result display)

Virsorter

quote from paper

3 Parameter: sure, somewhat sure, not so sure

for pro Phage: maybe generate a special output for tools that recognize pophages

• yes, maybe, no

deepvirfinder

quote from paper

We used deep learning techniques and developed a powerful framework for predicting viral sequences. Given aquery sequence, the framework gives a score between 0 and 1, and the larger score indicates the higher possibilityof being a viral sequence.

• p-value 0-1 gradation of result (other result display)

Metaphinder

• yes no result

[replikation]

@hoelzer we were thinking about changing the heatmap as a 2 color coded showing "certain" and "uncertain" hits

€ So the informations are not completely lost

[hoelzer]

@Stormrider935 cool thanks for the summary! Looks reasonable to me. And I think telling the user that he might not see a subset of the full output of all tools is important.

Changing the heat map to show "certain"/"uncertain" hits also sounds reasonable.

However, by thinking about the visualization of the results I remembered a cool plot I saw yesterday during a data visualization talk here at EBI (https://www.ebi.ac.uk/training/online/course/data-visualisation-101-practical-introduction-designing-scientific-figures)

Code: https://github.com/hms-dbmi/UpSetR

Couldn't this be a nice (additional) visualisation? I mean in the end you want to know how many phages were discovered by which tools and what is the overlap. Are there phages discovered by all tools? Are there single tools that discovered phages no other tool has? From such a plot this would be easy to see. Instead of Gene names like shown in the plot above, write the tool names?

[replikation]

the figure is nice, but how and what data should be included for this? i mean what should the barplot represent?

[replikation]

but yeah we could plot p values and such.

[hoelzer]

I thought that you can throw in a FASTA file and then you get phage predictions with different tools. Let's say

VirSorter: 20 phages VirFinder: 12 phages MARVEL: 30 phages

This are then the blue bars in the figure.

Then all overlapps (venn-diagram-style) are calculated, lets say:

VirSorter-VirFinder: 10 VirSorter-MARVEL: 18 VirFinder-MARVEL: 8 VirSorter-VirFinder-MARVEL: 6

And this could be then the black bar plots.

[replikation]

ah okay now i get it :) yeah with this plot it will be badass.

[hoelzer]

ah okay now i get it :) yeah with this plot it will be badass.

yeap, totally badass :D it's a super nice visualization alternative to venn diagrams (that one should anyway not use with more than 3 sets)

[hoelzer]

I think one open question might be how to define if two tools find the same phage. But maybe it's simple enough to check

tool A found contig X to be a phage tool B found contig X to be a phage

ok, both tools found a phage

[replikation]

when i have time ill try to implement something, and then we can take a look :)

[replikation]

my plan would be like this (x is a hit), not sure were to put the contig name.

                 x
                 xx
                 ____
    x metaphinderXO
      PPRmeta    OO
   xx Virsorter  XX

[hoelzer]

my plan would be like this (x is a hit), not sure were to put the contig name.

                 x
                 xx
                 ____
    x metaphinderXO
      PPRmeta    OO
   xx Virsorter  XX

yeah exactly what I thought. Unfortunately, I think in this visual the contig name can not be included... it would be then more like a general overview which tools find the most (common) phages

[hoelzer]

I added some example code for UpSetR here:

https://github.com/hoelzer/visual

maybe helpful if you want to add this to WtP at some point.

[hoelzer]

I fixed the docker for upsetr, its working now and implemented in branch upsetr. Please feel free to test @replikation @Stormrider935 and if satisfied, feel free to merge the branch ;)

At the moment the implementation is 'basic', so if new tools are added also the upset.R script needs to be updated. That could be enhanced with some R coding.

The upset plot shows the same content a venn diagram would show: simply by bars and easier to check for certain sets.

[replikation]

nice ill check that out

[replikation]

closing this, see #27 for performance