Tobias Snake make is Shutting down automatically

[mano2991]

I dont know why, My tobias snake make pipeline is automatically shuting down. I tried with the same sample multiple times, Each time its shutdown at different timepoint. I have enough space in my system space and memory. I cant able to figure out why this error is showin you can any one help me out. I have enlcosed my log file for your reference.

2023-11-06T155421.896425.snakemake.log

The command i used for the analysis. snakemake --configfile config.yaml --cores 30 --use-conda --conda-prefix Temp

[msbentsen]

Hi @mano2991,

Thank you for the log. It says that it is the format_motifs rule that fails - you can see the log for it at /home/manoj/Documents/anisha/DRG_simone/Tobias/logs/format_motifs.log.

I suspect the issue is due to a matplotlib deprecation error, which I just pushed a fix for today. If you run git pull or just download the TOBIAS_snakemake repository again, the error should be fixed. Please let me know if this is not the case.

BR Mette

[mano2991]

@msbentsen Still the error on matplotlib is ocuring.

TOBIAS DownloadData

Traceback (most recent call last):
  File "/home/manoj/anaconda3/envs/tobias_snakemake_env/bin/TOBIAS", line 7, in <module>
    from tobias.TOBIAS import main
  File "/home/manoj/anaconda3/envs/tobias_snakemake_env/lib/python3.10/site-packages/tobias/TOBIAS.py", line 38, in <module>
    warnings.filterwarnings("ignore", category=matplotlib.cbook.mplDeprecation)
AttributeError: module 'matplotlib.cbook' has no attribute 'mplDeprecation

While trying to run my sample Im facing an different error

[Wed Nov  8 10:46:17 2023]
Finished job 6.
4 of 14 steps (29%) done
Select jobs to execute...

[Wed Nov  8 10:46:17 2023]
rule uropa:
    input: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.config
    output: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits.txt, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.bed, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_header.txt
    log: /home/manoj/Documents/anisha/DRG_simone/Tobias/logs/uropa.log
    jobid: 5
    reason: Missing output files: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.bed, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_header.txt; Input files updated by another job: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.config
    threads: 30
    resources: tmpdir=/tmp

Activating conda environment: Temp/61c1023a82669b0f72b381fad05b3541_
cut: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits.txt: No such file or directory
[Wed Nov  8 10:46:34 2023]
Error in rule uropa:
    jobid: 5
    input: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.config
    output: /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits.txt, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.bed, /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_header.txt
    log: /home/manoj/Documents/anisha/DRG_simone/Tobias/logs/uropa.log (check log file(s) for error details)
    conda-env: /home/manoj/Documents/TOBIAS_snakemake/Temp/61c1023a82669b0f72b381fad05b3541_
    shell:
        uropa --input /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.config --prefix /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated --threads 30 &> /home/manoj/Documents/anisha/DRG_simone/Tobias/logs/uropa.log; cut -f 1-4,7-13,16-19 /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits.txt > /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt; head -n 1 /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt > /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_header.txt;tail -n +2 /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt > /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.bed
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job uropa since they might be corrupted:
/home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated_finalhits_sub.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-11-08T103458.327735.snakemake.log

I have enclosed the log file for your reference.

2023-11-08T103458.327735.snakemake.log

uropa.log

wilson_installation.log

[msbentsen]

Hi @mano2991 ,

Great, thank you for the logfiles. If you look into the uropa.log file, you can see the error:

2023-11-08 10:46:20 (485343) [INFO] Preparing .gtf-file for fast access
2023-11-08 10:46:34 (485343) [ERROR]    Query feature(s) ['gene'] not found in gtf

This means that there is no feature (3rd column of the .gtf) named gene to use for annotation. Can you check which .gtf-file you are given in the config file? Making sure that this has genes should solve the issue.

[mano2991]

Hello @msbentsen,

I'm currently using the mouse mm10 GTF file, which I downloaded from https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M22/gencode.vM22.annotation.gtf.gz.

While comparing this file with the example file from Tobias, I noticed that they appear quite similar. I've attached a screenshot of the first two lines from my GTF file for your reference.

In both files, the third column contains the value 'gene', and within this column, separated by semicolons, there are also gene names. Despite these similarities, I'm encountering an error in the TOBIAS pipeline, and I'm unsure why this is happening.

Additionally, it would be helpful if you could provide a link to download a GTF file that is compatible with the TOBIAS pipeline, or if you could specify the format requirements for a GTF file to be used with this tool.

[msbentsen]

Hi @mano2991 ,

Thank you for the link to the file. Are you giving it as .gtf or .gtf.gz in the config file? I will recommend to unzip it and give it as raw .gtf.

If this does not work, can you send me the content of the /home/manoj/Documents/anisha/DRG_simone/Tobias/peak_annotation/all_merged_annotated.config file? Thank you.

Best regards, Mette

[mano2991]

@msbentsen im using it in unziped format only. As i cant able to upload the config file directly, i have renamed it as .text for your refernce. all_merged_annotated.txt

[msbentsen]

@msbentsen im using it in unziped format only. As i cant able to upload the config file directly, i have renamed it as .text for your refernce. all_merged_annotated.txt

Great, that looks good. Can you check the output of: head -n 10 /home/manoj/Documents/anisha/DRG_simone/Tobias/flatfiles/mm10_v2.gtf and
tail -n 10 /home/manoj/Documents/anisha/DRG_simone/Tobias/flatfiles/mm10_v2.gtf

Just to make sure the .gtf file is not corrupted or otherwise has issues.

[mano2991]

@msbentsen, Identified the issue. I resolved it by removing the conda environment, reinstalling it, and restarting my machine, which helped clear the temporary memory. Thank you for your support