search

akcorut / kGWASflow

kGWASflow is a Snakemake workflow for performing k-mers-based GWAS.
https://github.com/akcorut/kGWASflow/wiki
MIT License
28 stars 8 forks source link

Error in rule create_kmers_table #12

Closed DavideGue closed 1 year ago

DavideGue commented 1 year ago

I'm trying to run kGWASflow starting from very few sample (4) to check memory requirements and got the following message: Error in rule create_kmers_table: jobid: 21 input: results/kmers_list/kmers_list_paths.txt, results/kmers_count/WB-001/kmers_with_strand, results/kmers_count/WB-002/kmers_with_strand, results/kmers_list/kmers_to_use, scripts/external/kmers_gwas/bin output: results/kmers_table/kmers_table.table log: logs/create_kmers_table/build_kmers_table.log (check log file(s) for error details) conda-env: /home/davideg/Desktop/kgwaswd/.snakemake/conda/37554b5fc2da0261ee9f3cdd28732365 shell:

    export LD_LIBRARY_PATH=$CONDA_PREFIX/lib

    scripts/external/kmers_gwas/bin/build_kmers_table -l results/kmers_list/kmers_list_paths.txt -k 31         -a results/kmers_list/kmers_to_use -o results/kmers_table/kmers_table 2> logs/create_kmers_table/build_kmers_table.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Error! The Snakemake workflow aborted. Complete log: .snakemake/log/2023-07-10T141439.391048.snakemake.log ERROR:root:Error running Snakemake: Command '['snakemake', '--use-conda', '--conda-frontend', 'conda', '--cores', '6', '--snakefile', '/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/Snakefile', '--rerun-triggers', 'mtime', 'params', 'input', 'software-env', 'code']' returned non-zero exit status 1.

I realized that the sorted kmer file is empty.

akcorut commented 1 year ago

Hi @DavideGue,

Thank you for using kGWASflow. Would you mind sharing the build_kmers_table.log and combine_and_filter.log files with me? You will find the first one in logs/create_kmers_table/ directory and the second one in logs/combine_and_filter/ directory.

I realized that the sorted kmer file is empty.

Which file exactly do you mean by his?

DavideGue commented 1 year ago

Hi @akcorut thank you for the fast reply. Attached you will find the files you requested. build_kmers_table.log combine_and_filter.log

I mean the file results/kmers_list/kmers_to_use

akcorut commented 1 year ago

Based on the log files, it looks like none of the k-mers passed the filtering step. This is most likely caused by sample size. 

kmers lower than MAC:   0
passed kmers:   0
passed MAC bot not pass strand filter:  0

With only 4 samples, even if you pass this step (creating a k-mers table), you won't be able to perform kmersGWAS. You'll need a minimum of ~30 samples.

Let me know if increasing the sample size fixes this issue.

DavideGue commented 1 year ago

I will try to follow your suggestions. BTW my experiment is a WES of 500 samples (25.000.000 reads/sample). What do you think is the minimum memory requirement to analyze the entire dataset?

DavideGue commented 1 year ago

I increased the sample size to 30 and after some samples were processed I received the following message: [Tue Jul 11 11:47:19 2023] Job 59: Creating symbolic links for fastq files... Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz Reason: Missing output files: results/reads/WB-025/WB-025_2.fastq.gz, results/reads/WB-025/WB-025_1.fastq.gz

Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz to results/reads/WB-027/WB-027_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz results/reads/WB-027/WB-027_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz to results/reads/WB-027/WB-027_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz results/reads/WB-027/WB-027_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log' returned non-zero exit status 1.

Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz results/reads/WB-024/WB-024_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz to results/reads/WB-024/WB-024_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz results/reads/WB-024/WB-024_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log' returned non-zero exit status 1.

[Tue Jul 11 11:47:19 2023] [Tue Jul 11 11:47:19 2023]

Error in rule create_symlink: jobid: 61 input: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz output: results/reads/WB-027/WB-027_1.fastq.gz, results/reads/WB-027/WB-027_2.fastq.gz log: logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz to results/reads/WB-027/WB-027_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz results/reads/WB-027/WB-027_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz to results/reads/WB-027/WB-027_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz results/reads/WB-027/WB-027_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Error in rule create_symlink: jobid: 58 input: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz output: results/reads/WB-024/WB-024_1.fastq.gz, results/reads/WB-024/WB-024_2.fastq.gz log: logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz results/reads/WB-024/WB-024_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz to results/reads/WB-024/WB-024_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz results/reads/WB-024/WB-024_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Tue Jul 11 11:47:19 2023] Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz to results/reads/WB-027/WB-027_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz results/reads/WB-027/WB-027_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz to results/reads/WB-027/WB-027_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz results/reads/WB-027/WB-027_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz to results/reads/WB-027/WB-027_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz results/reads/WB-027/WB-027_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz to results/reads/WB-027/WB-027_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz results/reads/WB-027/WB-027_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

Job 50: Creating symbolic links for fastq files... Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz results/reads/WB-024/WB-024_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz to results/reads/WB-024/WB-024_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz results/reads/WB-024/WB-024_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz results/reads/WB-024/WB-024_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz to results/reads/WB-024/WB-024_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz results/reads/WB-024/WB-024_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

Reason: Missing output files: results/reads/WB-016/WB-016_1.fastq.gz, results/reads/WB-016/WB-016_2.fastq.gz

Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz results/reads/WB-025/WB-025_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz to results/reads/WB-025/WB-025_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz results/reads/WB-025/WB-025_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log' returned non-zero exit status 1.

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz to results/reads/WB-027/WB-027_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_1P.fastq.gz results/reads/WB-027/WB-027_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz to results/reads/WB-027/WB-027_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-027.trimmomatic_2P.fastq.gz results/reads/WB-027/WB-027_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-027/WB-027_WB-027_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run [Tue Jul 11 11:47:19 2023] Error in rule create_symlink: jobid: 59 input: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz output: results/reads/WB-025/WB-025_1.fastq.gz, results/reads/WB-025/WB-025_2.fastq.gz log: logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz results/reads/WB-025/WB-025_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz to results/reads/WB-025/WB-025_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz results/reads/WB-025/WB-025_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz to results/reads/WB-024/WB-024_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_1P.fastq.gz results/reads/WB-024/WB-024_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz to results/reads/WB-024/WB-024_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-024.trimmomatic_2P.fastq.gz results/reads/WB-024/WB-024_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-024/WB-024_WB-024_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run [Tue Jul 11 11:47:19 2023] Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz results/reads/WB-025/WB-025_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz to results/reads/WB-025/WB-025_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz results/reads/WB-025/WB-025_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz results/reads/WB-025/WB-025_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz to results/reads/WB-025/WB-025_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz results/reads/WB-025/WB-025_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

Job 42: Creating symbolic links for fastq files... Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz Reason: Missing output files: results/reads/WB-008/WB-008_2.fastq.gz, results/reads/WB-008/WB-008_1.fastq.gz

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz to results/reads/WB-025/WB-025_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_1P.fastq.gz results/reads/WB-025/WB-025_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz to results/reads/WB-025/WB-025_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-025.trimmomatic_2P.fastq.gz results/reads/WB-025/WB-025_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-025/WB-025_WB-025_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz results/reads/WB-016/WB-016_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz to results/reads/WB-016/WB-016_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz results/reads/WB-016/WB-016_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log' returned non-zero exit status 1.

[Tue Jul 11 11:47:19 2023] Error in rule create_symlink: jobid: 50 input: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz output: results/reads/WB-016/WB-016_1.fastq.gz, results/reads/WB-016/WB-016_2.fastq.gz log: logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz results/reads/WB-016/WB-016_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz to results/reads/WB-016/WB-016_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz results/reads/WB-016/WB-016_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Tue Jul 11 11:47:19 2023] Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz Job 37: Creating symbolic links for fastq files... Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz results/reads/WB-016/WB-016_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz to results/reads/WB-016/WB-016_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz results/reads/WB-016/WB-016_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz results/reads/WB-016/WB-016_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz to results/reads/WB-016/WB-016_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz results/reads/WB-016/WB-016_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

Reason: Missing output files: results/reads/WB-003/WB-003_1.fastq.gz, results/reads/WB-003/WB-003_2.fastq.gz

Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz results/reads/WB-008/WB-008_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz to results/reads/WB-008/WB-008_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz results/reads/WB-008/WB-008_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log' returned non-zero exit status 1.

[Tue Jul 11 11:47:19 2023] Error in rule create_symlink: jobid: 42 input: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz output: results/reads/WB-008/WB-008_1.fastq.gz, results/reads/WB-008/WB-008_2.fastq.gz log: logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz results/reads/WB-008/WB-008_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz to results/reads/WB-008/WB-008_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz results/reads/WB-008/WB-008_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz to results/reads/WB-016/WB-016_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_1P.fastq.gz results/reads/WB-016/WB-016_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz to results/reads/WB-016/WB-016_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-016.trimmomatic_2P.fastq.gz results/reads/WB-016/WB-016_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-016/WB-016_WB-016_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz results/reads/WB-008/WB-008_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz to results/reads/WB-008/WB-008_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz results/reads/WB-008/WB-008_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz results/reads/WB-008/WB-008_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz to results/reads/WB-008/WB-008_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz results/reads/WB-008/WB-008_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz to results/reads/WB-008/WB-008_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_1P.fastq.gz results/reads/WB-008/WB-008_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz to results/reads/WB-008/WB-008_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-008.trimmomatic_2P.fastq.gz results/reads/WB-008/WB-008_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-008/WB-008_WB-008_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz results/reads/WB-003/WB-003_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz to results/reads/WB-003/WB-003_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz results/reads/WB-003/WB-003_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log' returned non-zero exit status 1.

[Tue Jul 11 11:47:19 2023] Error in rule create_symlink: jobid: 37 input: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz, /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz output: results/reads/WB-003/WB-003_1.fastq.gz, results/reads/WB-003/WB-003_2.fastq.gz log: logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log, logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log (check log file(s) for error details) shell:

        echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz
        echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz results/reads/WB-003/WB-003_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log
        echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz to results/reads/WB-003/WB-003_2.fastq.gz
        ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz results/reads/WB-003/WB-003_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Full Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2576, in run_wrapper run( File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 62, in rule_create_symlink script: File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/shell.py", line 300, in new__ raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz results/reads/WB-003/WB-003_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz to results/reads/WB-003/WB-003_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz results/reads/WB-003/WB-003_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 666, in _callback raise ex File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 652, in cached_or_run run_func(args) File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/snakemake/executors/init.py", line 2612, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz results/reads/WB-003/WB-003_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz to results/reads/WB-003/WB-003_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz results/reads/WB-003/WB-003_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink

RuleException: CalledProcessError in file /home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk, line 31: Command 'set -euo pipefail;
echo Working on fastq files: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz echo Symlink -fastq1: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz to results/reads/WB-003/WB-003_1.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_1P.fastq.gz results/reads/WB-003/WB-003_1.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_1.fastq.create_symlink.log echo Symlink -fastq2: /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz to results/reads/WB-003/WB-003_2.fastq.gz ln -rs /media/davideg/TOSHIBA EXT/WB-003.trimmomatic_2P.fastq.gz results/reads/WB-003/WB-003_2.fastq.gz 2> logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log' returned non-zero exit status 1. File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/rules/count_kmers.smk", line 31, in __rule_create_symlink File "/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/concurrent/futures/thread.py", line 58, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Error! The Snakemake workflow aborted. Complete log: .snakemake/log/2023-07-11T105206.169883.snakemake.log unlocking removing lock removing lock removed all locks ERROR:root:Error running Snakemake: Command '['snakemake', '--use-conda', '--conda-frontend', 'conda', '--cores', '6', '--snakefile', '/home/davideg/miniconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/Snakefile', '--rerun-triggers', 'mtime', 'params', 'input', 'software-env', 'code', '--verbose']' returned non-zero exit status 1.

akcorut commented 1 year ago

Can you share with me one of the log files inside logs/count_kmers/create_symlink/ directory? For example: logs/count_kmers/create_symlink/WB-003/WB-003_WB-003_2.fastq.create_symlink.log. It seems to be a symlink issue, but just to be sure.

DavideGue commented 1 year ago

It reports a problem in symlink, but I cannot understand why WB-003_WB-003_1.fastq.create_symlink.log

akcorut commented 1 year ago

Interesting. My guess is that there is a space in your path: /media/davideg/TOSHIBA EXT/. This can cause a problem. Is there any way you can change the name "TOSHIBA EXT/" and remove the space, then run the workflow again? Alternatively, you can move the input files somewhere else before running the pipeline.

DavideGue commented 1 year ago

After renaming the external disk it is running without any problem. I will wait the end of the analysis.

akcorut commented 1 year ago

Great to hear it is working now. Is it okay to close this issue now? You can always open a new one if you have any problems in the future.

DavideGue commented 1 year ago

Ok you can close it! Thank you for your time!