search

EI-CoreBioinformatics / mikado

Mikado is a lightweight Python3 pipeline whose purpose is to facilitate the identification of expressed loci from RNA-Seq data * and to select the best models in each locus.
https://mikado.readthedocs.io/en/stable/
GNU Lesser General Public License v3.0
94 stars 18 forks source link

Mikado Serialise error #323

Closed francicco closed 3 years ago

francicco commented 4 years ago

Hi Luca,

I'm getting this error during Mikado serialise, I don't know what it means , but afterwards mikado pick works and I have all the output:

mikado.subloci.out.gff3
mikado.subloci.out.scores.tsv
mikado.subloci.out.metrics.tsv
mikado.loci.gff3
mikado.loci.scores.tsv
mikado.loci.metrics.tsv
##### Fri Aug 14 09:35:47 BST 2020: Extract best CDS to Eisa.AugustusStringtie.transcripts.cds.fasta
Fri Aug 14 09:35:47 BST 2020: Mikado serialise
2020-08-14 09:36:19,144 - main - __init__.py:120 - ERROR - main - MainProcess - Mikado crashed, cause:
2020-08-14 09:36:19,144 - main - __init__.py:121 - ERROR - main - MainProcess - (sqlite3.OperationalError) no such index: orf_index
[SQL: 
DROP INDEX orf_index]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
Traceback (most recent call last):
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
    cursor, statement, parameters, context
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such index: orf_index

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/Mikado/__init__.py", line 106, in main
    args.func(args)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/Mikado/subprograms/serialise.py", line 381, in serialise
    load_orfs(args, logger)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/Mikado/subprograms/serialise.py", line 147, in load_orfs
    serializer()
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/Mikado/serializers/orf.py", line 507, in __call__
    [idx.drop(bind=self.engine) for idx in Orf.__table__.indexes]
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/Mikado/serializers/orf.py", line 507, in <listcomp>
    [idx.drop(bind=self.engine) for idx in Orf.__table__.indexes]
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 3928, in drop
    bind._run_visitor(ddl.SchemaDropper, self)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2098, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1657, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 144, in traverse_single
    return meth(obj, **kw)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 995, in visit_index
    self.connection.execute(DropIndex(index))
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1014, in execute
    return meth(self, multiparams, params)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1076, in _execute_ddl
    compiled,
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1318, in _execute_context
    e, statement, parameters, cursor, context
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1512, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
    cursor, statement, parameters, context
  File "/mnt/storage/scratch/tk19812/software/mikado/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such index: orf_index
[SQL: 
DROP INDEX orf_index]
(Background on this error at: http://sqlalche.me/e/13/e3q8)

As a side question: What's the difference between the loci and subloci output, which one should I use?

Thanks a lot for your support Cheers F

lucventurini commented 4 years ago

Hi @francicco ,

Well, this is a new error, I never saw this one. Might I ask for the full list of inputs and command line? Most importantly, did you provide ORFs derived from Augustus or are these transcripts with a CDS coming from Augustus?

I would not trust the results from pick, if serialise failed, pick will not have had available the extra data it needed (BLAST data, etc.). It probably used what it could, ie mostly transcript length ...

As a side question: What's the difference between the loci and subloci output, which one should I use?

Subloci and monosubloci are the intermediate steps during the Mikado algorithm, basically they are the first and second groupings. They can be ignored if not for debugging/development purposes (eg if you want to adapt the scoring file, or understand why a given transcript was discarded throughout the run).

The loci output is the one that should be used.

I hope this helps

Luca

francicco commented 4 years ago

Hi Luca

the command line is:

mikado serialise -p $THREADS --json-conf configuration.yaml --xml mikado.blast.xml.gz --orfs mikado_prepared.fasta.transdecoder.bed

ls -l mikado.blast.xml.gz mikado_prepared.fasta.transdecoder.bed
-rw-r--r-- 1 tk19812 bisc 90930120 Aug 14 02:00 mikado.blast.xml.gz
-rw-r--r-- 1 tk19812 bisc 34180769 Aug 14 08:27 mikado_prepared.fasta.transdecoder.bed

mikado_prepared.fasta.transdecoder.bed is the result of the TransDecoder pipeline on mikado_prepared.fasta. It includes a deltablast and hmmscan search on mikado_prepared.fasta.transdecoder_dir/longest_orfs.pep

F

lucventurini commented 4 years ago

Hi @francicco

OK, thank you. I will have a look at what might have caused the error.

Best

Luca