search

KarchinLab / open-cravat

A modular annotation tool for genomic variants
MIT License
110 stars 27 forks source link

Annotations missing from output #180

Closed tkmamidi closed 10 months ago

tkmamidi commented 12 months ago

I'm currently using v2.4.1 and some annotation columns are missing from the output when using only 1 CPU. Upon investigation, I found that aggregator step started before all the Annotators are finished running.

Here's an example - 

2023/08/24 00:26:56 cravat.ghis          started: Thu Aug 24 00:26:56 2023
2023/08/24 00:26:56 cravat.ghis          finished: Thu Aug 24 00:26:56 2023
2023/08/24 00:26:56 cravat.ghis          runtime: 0.027s
2023/08/24 00:26:56 cravat.ghis          started: Thu Aug 24 00:26:56 2023
2023/08/24 00:26:56 cravat.ghis          finished: Thu Aug 24 00:26:56 2023
2023/08/24 00:26:56 cravat.ghis          runtime: 0.001s
2023/08/24 00:26:56 cravat.mutation_assessor started: Thu Aug 24 00:26:56 2023
2023/08/24 00:26:58 cravat.aggregator    level: variant
2023/08/24 00:26:58 cravat.aggregator    input directory: /scratch/tmamidi/snvs/fa/6a8ebb0743dcc6b4a3355f02eaf3b3
2023/08/24 00:26:59 cravat.aggregator    started: Thu Aug 24 00:26:59 2023
2023/08/24 00:27:03 cravat.aggregator    finished: Thu Aug 24 00:27:03 2023
2023/08/24 00:27:03 cravat.aggregator    runtime: 4.653

Here, aggregator step started before mutation_assessor finished running.

Can someone please help me with solving this? Thank you!

tkmamidi commented 10 months ago

It is working well when used >=2 CPUs but still failing when using only 1 CPU.

kmoad commented 10 months ago

I've been able to recreate this and am working on a diagnosis. It seems like it's the combination of mutation_assessor and --mp 1. Multiprocessing issues are a bit tricky to diagnose, but I'll keep you updated as it's worked on.

kmoad commented 10 months ago

This turned out to be an issue with bothe the mutation_assessor annotator, and some poor error handling by the main package. The annotator is fixed by this commit https://github.com/KarchinLab/open-cravat-modules-karchinlab/commit/69c352d9dc5eb6baa4bbe049fcebd82387b11756.

Please update to mutation_assessor version 3.1.1 (oc module update mutation_assessor).

The error handling is fixed here https://github.com/KarchinLab/open-cravat/commit/d2b6b59400fac6fdd09373d09166ac84db7858c6 and that fix will be out in the next release. This will prevent similar errors from stopping annotator execution.