Using `flatfile-to-json.pl`: It apparently worked, but when creating the organism, nothing happened

[chibante]

Hi, guys!

I applied the flatfile-to-json.pl pipeline in my GFF3 file, apparently it was generated without problems. I created a new organism filling the usual fields, but after click on button, the data wasn't presented in the list of registered organisms. The conversion and creation of the organism in FASTA worked.

The command that I use: [apollo@mendel ~]$ screen -dmS GFF3 sudo perl /mnt/HD2/Apollo/jbrowse/bin/flatfile-to-json.pl --gff /home/apollo/meus_dados/GFF3/Bradysia_hygida/*.gff --type mRNA --trackLabel MAKER --out /mnt/HD2/Apollo/data/GFF3/Bhyg/

Thanks for help! :)

[nathandunn]

Could you supply a snippet of the gff3 and FASTA and any Javascript console output?

Nathan

On Mar 1, 2019, at 1:32 PM, chibante notifications@github.com wrote:

Hi, guys!

I applied the flatfile-to-json.pl pipeline in my GFF3 file, apparently it was generated without problems. I created a new organism filling the usual fields, but after click on button, the data wasn't presented in the list of registered organisms. The conversion and creation of the organism in FASTA worked.

The command that I use: [apollo@mendel ~]$ screen -dmS GFF3 sudo perl /mnt/HD2/Apollo/jbrowse/bin/flatfile-to-json.pl --gff /home/apollo/meus_dados/GFF3/Bradysia_hygida/*.gff --type mRNA --trackLabel MAKER --out /mnt/HD2/Apollo/data/GFF3/Bhyg/

Thanks for help! :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[chibante]

FASTA (it worked)

Part of FASTA file: (head 10)

>FBgn0033056 type=gene; loc=2R:complement(1944862..1947063); ID=FBgn0033056; name=CG7856; dbxref=FlyBase:FBgn0033056,FlyBase:FBan0007856,FlyBase_Annotation_IDs:CG7856,GB_protein:AAF57287,GB:AI945278,GB:AY113248,GB_protein:AAM29253,GB:BF499103,UniProt/TrEMBL:Q8MZC8,UniProt/TrEMBL:A1Z6J6,INTERPRO:IPR008957,OrthoDB5_Drosophila:EOG57WNH4,EntrezGene:35533,InterologFinder:35533,BIOGRID:61438,DroID:FBgn0033056,DRSC:FBgn0033056,FLIGHT:FBgn0033056,FlyAtlas:CG7856-RA,FlyMine:FBgn0033056,GenomeRNAi:35533,modMine:FBgn0033056; derived_computed_cyto=42A8-42A9%3B Limits computationally determined from genome sequence between @P{PZ}l(2)09851<up>09851</up>@%26@P{lacW}Src42A<up>k10108</up>@ and @P{lacW}l(2)k09848<up>k09848</up>@%26@P{EP}EP407@; gbunit=AE013599; MD5=7ca9c4371b97aaf7046cf83fefb65eb8; length=2202; release=r5.45; species=Dmel; 
GGTAAAGTTGCCTTGGCGTCAGTTGGCAGTTTGGGAAAAGCCTACACACT
TAATATTTCGATAGATACACTTATTTCGCAATCGTAGAAGATACCACAAA
TCTCTCTTCCGTAAATTATAAGTATGTCCAAGAGGGTGAGCATCATGTTG
CCCGACGAAATACCTGCGGCTCCGTCAGGCAGCAGGAGGAACCCGATGCC
CAGCCGCACACTCAAAAGTTCCATTAAAAGTGGATCAAGCGCTCGGACGC
AACCCGCACCGCCAGTGTTCGTGGTTGTCAACCCGGAGAGCAGCCAAAGC
TCGCTTACCTCGGGTCAGCGGAACAACACCAGCACGACCAGCAACAGTTC
CCTTAACCAACGAAATAAGGAGCACTCCGCGGCGAGGTTGTCAGTAACCG
AAGATACGGCAGCCCTGGACAGCAGTGCCTTGCTTCAAGGTGGATCCTCA
.
.
.

Filling fields in blank:

After click the Create Organism button, note that the organism is listed:

**GFF3*** (it not worked) - I tried this parameter --type mRNA and after this --type transcript

Part of GFF3 file: (head 10)

##gff-version 3
scf7180000105684    .   contig  1   7604    .   .   .   ID=scf7180000105684;Name=scf7180000105684
scf7180000105684    maker   gene    311 439 .   -   .   ID=genemark-scf7180000105684-processed-gene-0.0;Name=genemark-scf7180000105684-processed-gene-0.0
scf7180000105684    maker   mRNA    311 439 .   -   .   ID=genemark-scf7180000105684-processed-gene-0.0-mRNA-1;Parent=genemark-scf7180000105684-processed-gene-0.0;Name=genemark-scf7180000105684-processed-gene-0.0-mRNA-1;_AED=1.00;_eAED=1.00;_QI=0|-1|0|0|-1|1|1|0|43
scf7180000105684    maker   exon    311 439 .   -   .   ID=genemark-scf7180000105684-processed-gene-0.0-mRNA-1:exon:6492;Parent=genemark-scf7180000105684-processed-gene-0.0-mRNA-1
scf7180000105684    maker   CDS 311 439 .   -   0   ID=genemark-scf7180000105684-processed-gene-0.0-mRNA-1:cds;Parent=genemark-scf7180000105684-processed-gene-0.0-mRNA-1
scf7180000105684    genemark    match   311 439 .   -   .   ID=scf7180000105684:hit:25032:4.5.0.0;Name=genemark-scf7180000105684-abinit-gene-0.0-mRNA-1
scf7180000105684    genemark    match_part  311 439 .   -   .   ID=scf7180000105684:hsp:53296:4.5.0.0;Parent=scf7180000105684:hit:25032:4.5.0.0;Target=genemark-scf7180000105684-abinit-gene-0.0-mRNA-1 1 129 +;Gap=M129
###
scf7180000105684    blastn  expressed_sequence_match    2052    4583    146 +   .   ID=scf7180000105684:hit:25029:3.2.0.0;Name=TRINITY_DN73602_c0_g1_i1
.
.
.

Filling fields in blank:

After click the Create Organism button, note that the organism isn't listed:

[nathandunn]

I'm pretty sure its because your GFF3 points to a seqid (column 1 has scf7180000105684 in your example) that is different than your FASTA (FBgn0033056), though it would be helpful if you clicked on your tracks tab so I can see what you have.

It doesn't appear that you have any track turned on.

Also, if you could show me your network console in your browser that may help, but I think the problem is just misaligned GFF3 as stated above

[chibante]

Hi, Nathan!

How can I format my GFF3 file so that the Apollo application can display my gff data?

My Tracks tab is empty:

My Ref Sequence tab, only FASTA seqs is displayed:

Thanks for help! :)

[nathandunn]

This should be fine:

/mnt/HD2/Apollo/jbrowse/bin/flatfile-to-json.pl --gff /home/apollo/meus_dados/GFF3/Bradysia_hygida/*.gff --type mRNA --trackLabel MAKER --out /mnt/HD2/Apollo/data/GFF3/Bhyg/

Only point to a single GFF3 though (file.gff not *.gff) and make sure that column one corresponds to your scaffolds / chromosomes: https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md

[chibante]

Hi, Nathan!

We checked our file structure (bhyg_annoteted_geno_nomaked.all.gff) and this is according this https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md and we specified the file on command line screen -dmS GFF3 sudo perl /mnt/HD2/Apollo/jbrowse/bin/flatfile-to-json.pl --gff /home/apollo/meus_dados/GFF3/Bradysia_hygida/bhyg_annoteted_geno_nomaked.all.gff --type mRNA --trackLabel MAKER --out /mnt/HD2/Apollo/data/GFF3/Bhyg/

Do you suggest another test to check what's happend?

Thanks for help!

[nathandunn]

That command looks good.

1. Is nothing available in the Track panel to select?
2. Can you send me the output of that command?
3. Can you list the contents of /mnt/HD2/Apollo/data/GFF3/Bhyg/?
4. Can you upload the file /mnt/HD2/Apollo/data/GFF3/Bhyg/trackList.json?
5. Can you list the contents of /mnt/HD2/Apollo/data/GFF3/Bhyg/tracks
6. Can you provide any javascript console output on refreshing?
7. Can you provide any server log output (catalina.out typically) on refreshing?

Thanks

[chibante]

Answers:

1. Is nothing available in the Track panel to select?

No, in Track panel appears this:

2. Can you send me the output of that command?

screen -dmSL GFF3 sudo perl /mnt/HD2/Apollo/jbrowse/bin/flatfile-to-json.pl --gff /home/apollo/meus_dados/GFF3/Bradysia_hygida/bhyg_annoteted_geno_nomaked.all.gff --type mRNA --trackLabel MAKER --out /mnt/HD2/Apollo/data/GFF3/Bhyg/

on screen, inside file log nothing happend...

[apollo@mendel ~]$ tail -1 ~/screenlog.0 
[sudo] senha para apollo: 

...but this command generate this files and folders:

[apollo@mendel ~]$ ls -liah /mnt/HD2/Apollo/data/GFF3/Bhyg

total 20K
233119745 drwxr-xr-x. 3 root root 4,0K Mar  3 21:23 .
233119747 drwxr-xr-x. 4 root root 4,0K Mar 14 15:24 ..
233047432 -rw-r--r--. 1 root root  541 Mar  6 12:52 .htaccess
233051064 -rw-r--r--. 1 root root  399 Mar  6 12:53 trackList.json
233119746 drwxr-xr-x. 3 root root 4,0K Mar  3 21:22 tracks
233051063 -rw-r--r--. 1 root root    0 Mar  3 21:23 tracks.conf

3. Can you list the contents of /mnt/HD2/Apollo/data/GFF3/Bhyg/ ?

[apollo@mendel ~]$ ls -liah /mnt/HD2/Apollo/data/GFF3/Bhyg

total 20K
233119745 drwxr-xr-x. 3 root root 4,0K Mar  3 21:23 .
233119747 drwxr-xr-x. 4 root root 4,0K Mar 14 15:24 ..
233047432 -rw-r--r--. 1 root root  541 Mar  6 12:52 .htaccess
233051064 -rw-r--r--. 1 root root  399 Mar  6 12:53 trackList.json
233119746 drwxr-xr-x. 3 root root 4,0K Mar  3 21:22 tracks
233051063 -rw-r--r--. 1 root root    0 Mar  3 21:23 tracks.conf

4. Can you upload the file /mnt/HD2/Apollo/data/GFF3/Bhyg/trackList.json ?

{
   "formatVersion" : 1,
   "tracks" : [
      {
         "compress" : 0,
         "key" : "MAKER",
         "label" : "MAKER",
         "storeClass" : "JBrowse/Store/SeqFeature/NCList",
         "style" : {
            "className" : "feature"
         },
         "trackType" : null,
         "type" : "FeatureTrack",
         "urlTemplate" : "tracks/MAKER/{refseq}/trackData.json"
      }
   ]
}

5. Can you list the contents of /mnt/HD2/Apollo/data/GFF3/Bhyg/tracks ?

[apollo@mendel ~]$ ls -liah /mnt/HD2/Apollo/data/GFF3/Bhyg/tracks
total 528K
233119746 drwxr-xr-x.     3 root root 4,0K Mar  3 21:22 .
233119745 drwxr-xr-x.     3 root root 4,0K Mar  3 21:23 ..
233144325 drwxr-xr-x. 15375 root root 516K Mar  6 12:53 MAKER

And in MAKER folder exist too much folders like this...

...
236363807 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112068
236363808 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112070
236363809 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112071
236363810 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112077
236363811 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112079
236363812 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112080
236363813 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112081
236363814 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112083
236363815 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112084
236363816 drwxr-xr-x.     2 root root 4,0K Mar  6 12:53 scf7180000112085

6. Can you provide any javascript console output on refreshing?

7. Can you provide any server log output (catalina.out typically) on refreshing?

Like this?

[apollo@mendel ~]$ sudo tail -f /var/log/tomcat/catalina.out

empty...

Thanks for your help! Carla

[nathandunn]

@chibante This all looks good.

However, for 5. you have javascript network selected (that all looks good).

1. Will probably also need to have need to have xhr (or just All) selected.

2. Will also need javascript console (on the upper-left, instead of network).

That being said, something weird is going on:

1. your trackList.json doesn't seem to have a sequence track listed? I'm guessing the directory you think you are pointing to is incorrect. Can you show me your organism panel with the directory for that organism shown?

If that directory is different, you'll need to show contents of that directory and merge what you have in the trackList.json.

[chibante]

Sorry...now, my console java:

My Organism panel:

I'm using Docker, this is a problem?

My Docker volumes configuration:

[apollo@mendel ~]$ sudo docker inspect -f "{{json .Mounts}}" APOLLO-PRODUCTION | jq .

[
  {
    "Type": "bind",
    "Source": "/mnt/HD2/Apollo/data",
    "Destination": "/data",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
  },
  {
    "Type": "bind",
    "Source": "/mnt/HD2/Apollo/postgres-data",
    "Destination": "/var/lib/postgresql",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
  }
]

[nathandunn]

Docker should be fine (or if it fails, it would fail catastrophically).

Looking here:

The names in the organism panel don't match the name in your UI fields. Did you ever hit Create Organism?

If so, what happened?

[chibante]

Dmel and Dmel2 are organisms created by FAST files on another occasion and worked.

Yes, I clicked on . The same way I did to register files from FAST (Dmel and Dmel2).

[nathandunn]

can you do that again and show me what you have?