GFF generated from mummer: aggregate scaffolds into a single parent feature

[bradfordcondon]

Feature description

The walnut GFF tracks loaded in consist of lots of little, disconnected features. #227
I need to create a parent track, per scaffold, so that it will display cleaner and be more clear.

Link to relevant info

https://github.com/bradfordcondon/simple_biopython/blob/master/staton_biopy/coords_to_gff.py

[bradfordcondon]

gff parser improved (see https://github.com/bradfordcondon/simple_biopython/issues/1).

Dev server currently being copied over so cant test if we've achieved the desired result until tomorrow.

[bradfordcondon]

hmmm terms with no ID have a parent that wasnt created.

sudo ../jbrowse/bin/flatfile-to-json.pl --gff stevens_gff_v2/wingnut_chandler_v2.gff --tracklabel p_stenoptera_new_test

GFF3 parse error: some features reference other features that do not exist in the file (or in the same '###' scope).  A list of them:
 ID                 |           Cannot Find
----------------------------------------------------------------------
(no id)             | Parent=C840987
(no id)             | Parent=C840987
 at /home/data/apollo/jbrowse/bin/../src/perl5/Bio/JBrowse/ConfigurationManager.pm line 7.
    Bio::JBrowse::ConfigurationManager::__ANON__('\x{a}GFF3 parse error: some features reference other features tha...') called at /home/data/apollo/jbrowse/bin/../src/perl5/../../extlib/lib/perl5/Bio/GFF3/LowLevel/Parser.pm line 195
    Bio::GFF3::LowLevel::Parser::_buffer_all_under_construction_features('Bio::GFF3::LowLevel::Parser=HASH(0x2096eb8)') called at /home/data/apollo/jbrowse/bin/../src/perl5/../../extlib/lib/perl5/Bio/GFF3/LowLevel/Parser.pm line 168
    Bio::GFF3::LowLevel::Parser::_buffer_items('Bio::GFF3::LowLevel::Parser=HASH(0x2096eb8)') called at /home/data/apollo/jbrowse/bin/../src/perl5/../../extlib/lib/perl5/Bio/GFF3/LowLevel/Parser.pm line 73
    Bio::GFF3::LowLevel::Parser::next_item('Bio::GFF3::LowLevel::Parser=HASH(0x2096eb8)') called at /home/data/apollo/jbrowse/bin/../src/perl5/Bio/JBrowse/FeatureStream/GFF3_LowLevel.pm line 16
    Bio::JBrowse::FeatureStream::GFF3_LowLevel::next_items('Bio::JBrowse::FeatureStream::GFF3_LowLevel=HASH(0x1fa0800)') called at /home/data/apollo/jbrowse/bin/../src/perl5/Bio/JBrowse/Cmd/NCFormatter.pm line 52
    Bio::JBrowse::Cmd::NCFormatter::_format('Bio::JBrowse::Cmd::FlatFileToJson=HASH(0x17c6150)', 'trackConfig', 'HASH(0x19b0c30)', 'featureStream', 'Bio::JBrowse::FeatureStream::GFF3_LowLevel=HASH(0x1fa0800)', 'featureFilter', 'CODE(0x1f99778)', 'trackLabel', 'p_stenoptera_new_test', ...) called at /home/data/apollo/jbrowse/bin/../src/perl5/Bio/JBrowse/Cmd/FlatFileToJson.pm line 128
    Bio::JBrowse::Cmd::FlatFileToJson::run('Bio::JBrowse::Cmd::FlatFileToJson=HASH(0x17c6150)') called at ../jbrowse/bin/flatfile-to-json.pl line 9

I can confirm that for some reason, this scaffold wasnt written to the GFF as a parent.

[bradfordcondon]

fixed a problem in hte script that printe d the previous parent instead of the current parent. I'll wait to resubmit until the sync finishes though.

[bradfordcondon]

I also fixed a problem where the parent feature start/end coordinates wasnt set properly.

I've uploaded to dev and added a track to test. Need to fix permissions #314 before I can verify.

[almasaeed2010]

314 is now fixed.

[bradfordcondon]

https://hardwoods.ag.utk.edu/tools/jbrowse/index.html?data=english_walnut&loc=chloroplast%3A81637..83544&tracks=p_stenoptera_new_test_v%2Cgene&highlight=

• the browser is SLOW now. its too much data loaded.
• Styling doesnt display each alignment chunk as a segmented piece, a la the exons in the gene track below. This could help with legibility but probably not by much. (this should be achievable by setting the "subParts" flag to "alignment")

Steps forward

• Further refine script. Right now, it creates one feature per ref/query scaffold combo. So if you align at start and end, you'll have a single feature that spans the whole reference.
  • Could resolve by setting arbitrary distance cutoff, after which we create a new parent feature.
• filter coords file by size or minimum pooled alignment length?

[bradfordcondon]

the above image is looking at the most problematic scaffold: chloroplast.

or

https://hardwoods.ag.utk.edu/tools/jbrowse/index.html?data=english_walnut&loc=jcf7180001221206%3A1..358591&tracks=p_stenoptera_new_test_v%2Cgene&highlight=

I still think the problem of a hit at the start/end of the scaffold will cause the alignment to span the whole thing, not showing a connected region with black line in the middle.

Also, filtering by size of the total parent could very nicely remove all the repeat stuff.

[mestato]

@mestato ask Jiali to rerun nucmer with these genomes and see if we can get a more sensible gff

[bradfordcondon]

• We are hoping that more rigorous nucmer alignment can resolve this issue.