search

TORCH-Consortium / MAGMA

A pipeline for comprehensive genomic analyses of Mycobacterium tuberculosis with a focus on clinical decision making as well as research
https://doi.org/10.1371/journal.pcbi.1011648
GNU General Public License v3.0
12 stars 2 forks source link

MINOR_VARIANTS_ANALYSIS_WF:TBPROFILER_VCF_PROFILE__LOFREQ error on conda config #207

Open Mxrcon opened 1 month ago

Mxrcon commented 1 month ago

Hey there :wave:, I just found an error when running latest version on local conda version. Aparently the process TBPROFILER_VCF_PROFILE_LOFREQ is using the incorrect conda env "conda_envs/magma-env-1" when it should be "conda_envs/magma-tbprofiler-env"

Here is the full log:

  error [nextflow.exception.ProcessFailedException]: Process `MINOR_VARIANTS_ANALYSIS_WF:TBPROFILER_VCF_PROFILE__LOFREQ (joint)` terminated with an error exit status (1)
Jul-11 14:08:04.044 [TaskFinalizer-5] ERROR nextflow.processor.TaskProcessor - Error executing process > 'MINOR_VARIANTS_ANALYSIS_WF:TBPROFILER_VCF_PROFILE__LOFREQ (joint)'

Caused by:
  Process `MINOR_VARIANTS_ANALYSIS_WF:TBPROFILER_VCF_PROFILE__LOFREQ (joint)` terminated with an error exit status (1)

Command executed:

  bcftools view joint.lofreq.vcf.gz | sed 's/NC-000962-3-H37Rv/Chromosome/g' > intermediate.vcf

  cat  intermediate.vcf | bcftools view -Oz -o intermediate.vcf.gz

  tb-profiler profile \
       \
      --threads 3\
      --vcf intermediate.vcf.gz \
       --depth 0,0 --af 0,0 --strand 0 --sv_depth 0,0 --sv_af 0,0 --sv_len 100000,50000

Command exit status:
  1

Command error:
  [14:08:03] INFO     Using ref file: /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.fasta     db.py:795
             INFO     Using gff file: /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.gff       db.py:795
             INFO     Using bed file: /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.bed       db.py:795
             INFO     Using version file:                                                                                        db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.version.json                       
             INFO     Using json_db file:                                                                                        db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.dr.json                            
             INFO     Using variables file:                                                                                      db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.variables.json                     
             INFO     Using spoligotype_spacers file:                                                                            db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.spoligotype_spacers.txt            
             INFO     Using spoligotype_annotations file:                                                                        db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.spoligotype_list.csv               
             INFO     Using bedmask file:                                                                                        db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.mask.bed                           
             INFO     Using barcode file:                                                                                        db.py:795
                      /data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/share/tbprofiler//tbdb.barcode.bed                        
  Traceback (most recent call last):
    File "/data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/bin/tb-profiler", line 559, in <module>
      args.func(args)
    File "/data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/bin/tb-profiler", line 84, in main_profile
      args.conf['variant_filters'] = pp.get_variant_filters(args)
    File "/data/davimarcon/PIBIC-Danna/MAGMA/conda_envs/magma-env-1/lib/python3.10/site-packages/pathogenprofiler/cli.py", line 21, in get_variant_filters
      filters[f+"_soft"] = float(vals[1]) if "." in vals[1] else int(vals[1])
  IndexError: list index out of range
  Cleaning up after failed run
             ERROR                                                                                                          tb-profiler:58

In my opinion this issue is caused by 'LOFREQ.' and 'TBPROFILER.' selectors overlapping, my quick fix was to change the order of how the settings are imported.

I'm running it locally on a computer, without using any different configuration rather than conda_local and laptop. For any necessary request I'll be available to provide more information.

Kindly, Davi

abhi18av commented 1 month ago

Thanks for reporting this Davi, I think the comment https://github.com/TORCH-Consortium/MAGMA/pull/208#pullrequestreview-2174196210 might clarify the overall problem.

The conda envs were out of sync with the latest design, where we have isolated tb-profiler and ntm-profiler work.

And I've added a temporary misc container to test any quick changes, before rebuilding the magma-container-1 and magma-container-2 (to avoid breaking cache) for larger runs.