Records dropped as duplicates but then 'added back' as types are absent from named_type_hits output

[dhoogest]

Looks like there's a bit of a circular issue emerging from interplay between definition of the named .fasta set (which has duplicate records within a genome dropped https://github.com/nhoffman/ya16sdb/blob/master/SConstruct#L496), and the logic which adds all type strain records back to the 'trusted' .fasta output (and BLAST db). The outcome of this is that the trusted BLASTdb contains dropped duplicate alleles for some seqs within is_type genomes, and these records lack info about the nearest type strain, since the named fa is used as a target in https://github.com/nhoffman/ya16sdb/blob/master/SConstruct#L737

Possible solutions:

• build 'named_type_hits' from the trusted fa, either instead of or in addition to the current vsearch output
• add logic to prevent drop of duplicate alleles from type records
• move deduplication step in partition_refs https://github.com/nhoffman/ya16sdb/blob/master/bin/partition_refs.py#L116 after the append of trusted records

The third option seems easiest implementation-wise.

[dhoogest]

Example accession NZ_CP056776_2309782_2311321 which user visualized in NGS16S validation here as lacking 'closest type' info:

dhoogest@naga:$ grep NZ_CP056776_2309782_2311321 /molmicro/common/ncbi/16s/output/20211004/dedup/1200bp/named/seqs.fasta | wc -l 
0

No records for this allele in the 'named' set, its a duplicate allele of another for the NZ_CP056776 genome

Examining the 'trusted' set confirms the presence of the record seqs.fasta (which would also be the target for BLAST db used in the pipeline output where bug was detected).

dhoogest@naga:$ grep NZ_CP056776_2309782_2311321 /molmicro/common/ncbi/16s/output/20211004/dedup/1200bp/named/filtered/trusted/seqs.fasta | wc -l
1

[crosenth]

I believe this if fixed right?