Glimmer extract fails due to missing files

[dleehr]

When running example control file, the Glimmer extract stage fails because the following files are not produced:

• MMAP_example.glimmer.detail
• MMAP_example.glimmer.predict

However, glimmer extract does produce

• MMAP_example.glimmer.run1.detail
• MMAP_example.glimmer.run1.predict

$ python src/core/main.py /vagrant/control 
NumPy version 1.8.1
SciPy version 0.13.3
Bio version 1.63 at ['/usr/lib/python2.7/dist-packages/Bio']
Parsing control file: /vagrant/control
Running Genovo assemble...
Running Genovo finalize...
Running Glimmer...
Running Glimmer extract...
FileNotFound /vagrant/MMAP/data/example/MMAP_example.glimmer.detail
FileNotFound /vagrant/MMAP/data/example/MMAP_example.glimmer.predict
Traceback (most recent call last):
  File "src/core/main.py", line 83, in <module>
    assembler.run()
  File "/vagrant/MMAP/src/core/assembler/software_assembler.py", line 114, in run
    self.glimmer.run(self.debug)
  File "/vagrant/MMAP/src/core/component/run_glimmer.py", line 115, in run
    self._isCompleted()
  File "/vagrant/MMAP/src/core/component/run_glimmer.py", line 120, in _isCompleted
    raise(StandardError("Glimmer did not complete, not all output files exist"))
StandardError: Glimmer did not complete, not all output files exist

[dleehr]

I propose removing the .detail and .predict entries from run_glimmer.py, leaving .run1.detail and .run1.predict.

[stevenhwu]

This one is interesting, I suspect another bug at my end. According to glimmer doc. .detail and .predict should be the final output. I am going through glimmer again.

"<tag>.detail is the final Glimmer3 output detail"
"<tag>.predict is the final Glimmer3 predictions"

If this is not too much work, can you please run it under the debug mode and paste the output here.

python src/core/main.py -d /vagrant/control > debug.log

[dleehr]

I posted my debug.log, but I think the relevant part is here:

Running Glimmer...
debug: _command:    ['./g3-iterated.csh_linux', '/vagrant/MMAP/data/example/MMAP_example.genovo', '/vagrant/MMAP/data/example/MMAP_example.glimmer']
debug - output message:
Step 1 of 8:  Finding long orfs for training
Step 2 of 8:  Extracting training sequences
Step 3 of 8:  Building ICM
Step 4 of 8:  Running first Glimmer3
Step 5 of 8:  Getting training coordinates
Step 6 of 8:  Making PWM from upstream regions
Failed to create PWM

errors message:
Starting at Fri Feb 20 20:08:23 2015

Sequence file = /vagrant/MMAP/data/example/MMAP_example.genovo
Excluded regions file = none
Circular genome = true
Initial minimum gene length = 90 bp
Determine optimal min gene length to maximize number of genes
Maximum overlap bases = 30
Start codons = atg,gtg,ttg
Stop codons = taa,tag,tga
Sequence length = 1554
Final minimum gene length = 94
Number of genes = 2
Total bases = 285
Starting at Fri Feb 20 20:08:23 2015

Sequence file = /vagrant/MMAP/data/example/MMAP_example.genovo
Number of sequences = 1090
ICM model file = /vagrant/MMAP/data/example/MMAP_example.glimmer.icm
Excluded regions file = none
List of orfs file = none
Input is NOT separate orfs
Independent (non-coding) scores are used
Circular genome = true
Truncated orfs = false
Minimum gene length = 110 bp
Maximum overlap bases = 50
Threshold score = 30
Use first start codon = false
Start codons = atg,gtg,ttg
Start probs = 0.600,0.300,0.100
Stop codons = taa,tag,tga
GC percentage = 50.6%
Ignore score on orfs longer than 747
Analyzing Sequence #1
Start Find_Orfs
Start Score_Orfs
Start Process_Events
Start Trace_Back
Analyzing Sequence #2
...
Analyzing Sequence #1090
Start Find_Orfs
Start Score_Orfs
Start Process_Events
Start Trace_Back
/usr/bin/elph: Command not found.

Looks like we're missing elph. I see that Glimmer Release Notes mentions it's used by some of the csh scripts, but it's not in the repo.

[dleehr]

Yeah, once I installed ELPH and made it available as /usr/bin/elph, this worked. ELPH is required by the g3-iterated.csh script.

[stevenhwu]

Thank you Dan, and I think you actually found a real bug in my code. Working on fixing that now.

[dleehr]

This could be closed as invalid, unless you want to reuse the issue for the bug you discovered.