Hi,

I tried creating a new custom database, but when running the pipeline I get the following error:

Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
    1   all
    1   clark_db_custom_1
    1   clark_db_custom_2
    1   clark_db_custom_3
    1   clark_db_custom_check
    1   clark_db_custom_profile
    1   clark_rpt
    1   clark_run_1
    3   clean_files
    1   clean_reads
    3   database_profile
    1   kaiju_db_custom_2
    1   kaiju_db_custom_3
    1   kaiju_db_custom_check
    1   kaiju_db_custom_profile
    1   kaiju_rpt
    1   kaiju_run_1
    1   kraken_db_custom_1
    1   kraken_db_custom_3
    1   kraken_db_custom_check
    1   kraken_db_custom_profile
    1   kraken_rpt
    1   kraken_run_1
    1   krona
    1   metametamerge
    29
rule kaiju_db_custom_2:
    input: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa
    output: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.bwt, /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.sa
    log: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.log
    benchmark: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.time
    wildcards: database=new_custom_fungi_viral_db
    threads: 12

rule kraken_db_custom_1:
    output: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kraken_db/library/
    log: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kraken_db_custom_1.log
    benchmark: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kraken_db_custom_1.time
    wildcards: database=new_custom_fungi_viral_db
    threads: 12

rule kaiju_db_custom_profile:
    output: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju.dbaccession.out
    log: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_profile.log
    benchmark: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_profile.time
    wildcards: database=new_custom_fungi_viral_db

rule clark_db_custom_profile:
    output: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark.dbaccession.out
    log: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/clark_db_custom_profile.log
    benchmark: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/clark_db_custom_profile.time
    wildcards: database=new_custom_fungi_viral_db

rule clark_db_custom_1:
    output: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark_db/Custom/
    log: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/clark_db_custom_1.log
    benchmark: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/clark_db_custom_1.time
    wildcards: database=new_custom_fungi_viral_db

grep: custom_fungi_viral_db/kaiju//*.gbff: No such file or directory
grep: custom_fungi_viral_db/clark_dudes//*.fna: No such file or directory
cat: custom_fungi_viral_db/clark_dudes//*.fna: No such file or directory
Error in job kaiju_db_custom_profile while creating output file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju.dbaccession.out.
Error in job clark_db_custom_1 while creating output file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark_db/Custom/.
Error in job clark_db_custom_profile while creating output file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark.dbaccession.out.
RuleException:
CalledProcessError in line 44 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm:
Command 'grep -h '^VERSION' custom_fungi_viral_db/kaiju//*.gbff | tr -s ' ' | cut -d ' ' -f 2 > /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju.dbaccession.out 2> /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_profile.log' returned non-zero exit status 2
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm", line 44, in __rule_kaiju_db_custom_profile
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
RuleException:
CalledProcessError in line 69 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/clark_db_custom.sm:
Command 'grep -h '^>' custom_fungi_viral_db/clark_dudes//*.fna | grep -o '[A-Z]*_[0-9]*\.[0-9]*' | sed 's/>//g' > /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark.dbaccession.out 2> /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/clark_db_custom_profile.log' returned non-zero exit status 1
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/clark_db_custom.sm", line 69, in __rule_clark_db_custom_profile
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job kaiju_db_custom_profile since they might be corrupted:
/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju.dbaccession.out
RuleException:
CalledProcessError in line 7 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/clark_db_custom.sm:
Command '
        # Separate one file per sequence
        mkdir -p /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark_db/Custom/
        cat custom_fungi_viral_db/clark_dudes//*.fna | awk -v sep_seq_folder='/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark_db/Custom/' '{if (substr($0, 1, 1)==">") {filename=(sep_seq_folder "/" substr($1,2) ".fna")}; print $0 > filename}'
        ' returned non-zero exit status 1
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/clark_db_custom.sm", line 7, in __rule_clark_db_custom_1
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job clark_db_custom_profile since they might be corrupted:
/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark.dbaccession.out
Removing output files of failed job clark_db_custom_1 since they might be corrupted:
/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/clark_db/Custom/
Job failed, going on with independent jobs.
Job failed, going on with independent jobs.
Job failed, going on with independent jobs.
Error in job kaiju_db_custom_2 while creating output files /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.bwt, /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.sa.
RuleException:
CalledProcessError in line 17 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm:
Command 'mkbwt -n 5 -a ACDEFGHIKLMNPQRSTVWY -nThreads 12 -o /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa > /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.log 2>&1' returned non-zero exit status 5
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm", line 17, in __rule_kaiju_db_custom_2
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Job failed, going on with independent jobs.
ls: cannot access custom_fungi_viral_db/kraken//*.fna: No such file or directory
Error in job kraken_db_custom_1 while creating output file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kraken_db/library/.
RuleException:
CalledProcessError in line 8 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kraken_db_custom.sm:
Command 'ls -t custom_fungi_viral_db/kraken//*.fna | xargs --max-procs=12 -I '{}' kraken-build --db /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kraken_db/ --add-to-library '{}' >> /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kraken_db_custom_1.log 2>&1' returned non-zero exit status 2
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kraken_db_custom.sm", line 8, in __rule_kraken_db_custom_1
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job kraken_db_custom_1 since they might be corrupted:
/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kraken_db/library/
Job failed, going on with independent jobs.
Exiting because a job execution failed. Look above for error message

---------------------------------------------------------------------------------------
MetaMeta Pipeline v1.1 by Vitor C. Piro (vitorpiro@gmail.com, http://github.com/pirovc)
---------------------------------------------------------------------------------------
Parameters:
 - archaea_bacteria: {'clark': 'https://zenodo.org/record/820055/files/clark_bac_arc_v1.tar.gz',
 'dudes': 'https://zenodo.org/record/820053/files/dudes_bac_arc_v1.tar.gz',
 'gottcha': 'https://zenodo.org/record/819341/files/gottcha_bac_arc_v1.tar.gz',
 'kaiju': 'https://zenodo.org/record/819425/files/kaiju_bac_arc_v1.tar.gz',
 'kraken': 'https://zenodo.org/record/819363/files/kraken_bac_arc_v1.tar.gz',
 'motus': 'https://zenodo.org/record/819365/files/motus_bac_arc_v1.tar.gz'}
 - bins: 4
 - cutoff: 0.0001
 - databases: ['new_custom_fungi_viral_db']
 - dbdir: '/home/stephan/work/epityp/mirnaseq/05_metameta/databases/'
 - desiredminlen: 70
 - detailed: 0
 - errorcorr: 0
 - gzipped: 1
 - keepfiles: 1
 - mode: 'linear'
 - new_custom_fungi_viral_db: {'clark': 'custom_fungi_viral_db/clark_dudes/',
 'dudes': 'custom_fungi_viral_db/clark_dudes/',
 'kaiju': 'custom_fungi_viral_db/kaiju/',
 'kraken': 'custom_fungi_viral_db/kraken/'}
 - ranks: 'species'
 - replacement: 0
 - samples: {'sample_name_1': {'fq1': '/home/stephan/work/epityp/mirnaseq/05_metameta/input/not_mapped.fastq.gz'}}
 - samplesize: 1
 - strictness: 0.8
 - subsample: 0
 - threads: 12
 - tool_alt_path: {'bowtie2': '',
 'clark': '',
 'dudes': '',
 'gottcha': '',
 'kaiju': '',
 'kraken': '',
 'krona': '',
 'metametamerge': '',
 'motus': '',
 'spades': '',
 'trimmomatic': ''}
 - tools: {'clark': 'b', 'gottcha': 'p', 'kaiju': 'b', 'kraken': 'b', 'motus': 'p'}
 - trimming: 0
 - verbose: 0
 - workdir: '/home/stephan/work/epityp/mirnaseq/05_metameta/output/'
---------------------------------------------------------------------------------------

yaml


/home/stephan/work/epityp/mirnaseq/05_metameta/epitype_test.yaml                                                                                                                                                   1361/1361              100%
workdir: "/home/stephan/work/epityp/mirnaseq/05_metameta/output/"

Database output directory (Tip: create this folder in a common directory so it could be used for other runs as well as other users)

dbdir: "/home/stephan/work/epityp/mirnaseq/05_metameta/databases/"

Sample (name and files)

samples: "sample_name_1": fq1: "/home/stephan/work/epityp/mirnaseq/05_metameta/input/not_mapped.fastq.gz"

Add more samples here

Custom database

databases:

"new_custom_fungi_viral_db"

"new_custom_fungi_viral_db": clark: "custom_fungi_viral_db/clark_dudes/" dudes: "custom_fungi_viral_db/clark_dudes/" kaiju: "custom_fungi_viral_db/kaiju/" kraken: "custom_fungi_viral_db/kraken/"

################################################################

Configured tools (p=profiling, b=binning) from tools folder (tool.sm and tool_db.sm)

tools: "clark": "b"

"dudes": "p"

"gottcha": "p"
"kaiju": "b"
"kraken": "b"
"motus": "p"

MetaMeta Pipeline

Number of threads for each tool (distributed among the number of cores defined by main parameter --cores)

threads: 12

Gzipped input files (0: not gzipped / 1: gzipped). Default: 0

gzipped: 1

Keep intermediate files (database, reads and output) (0: do not keep files / 1: keep all files). Default: 0

keepfiles: 1 ### TODO change to 0



Thanks,
Stephan

Hi Stephan,

It looks like MetaMeta couldn't find the sequence files on the following paths: custom_fungi_viral_db/kaiju/*.gbff custom_fungi_viral_db/clark_dudes/*.fna

Any path or file in the .yaml file is always relative to the working directory set on the workdir variable (in this case /home/stephan/work/epityp/mirnaseq/05_metameta/output/). You can move the folder custom_fungi_viral_db to this path or use the complete path to indicate where are the sequences.

Best, Vitor

Hi,

I could now build the databases but during the first run I get this error:

Error in job kaiju_db_custom_2 while creating output files /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.bwt, /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi
_viral_db/kaiju_db/kaiju_db.sa.
RuleException:
CalledProcessError in line 17 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm:
Command 'mkbwt -n 5 -a ACDEFGHIKLMNPQRSTVWY -nThreads 12 -o /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_vir
al_db/kaiju_db/kaiju_db.faa > /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.log 2>&1' returned non-zero exit status 5
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm", line 17, in __rule_kaiju_db_custom_2
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Job failed, going on with independent jobs.
.[33mUnable to set utime on symlink sample_name_1/reads/pre1.1.fq.gz. Your Python build does not support it..[0m
Unable to set utime on symlink sample_name_1/reads/pre1.1.fq.gz. Your Python build does not support it.
1 of 19 steps (5%) done
rule errorcorr_reads:
    input: sample_name_1/reads/pre1.1.fq.gz
    output: sample_name_1/reads/pre2.1.fq.gz
    log: sample_name_1/log/errorcorr_reads.log
    benchmark: sample_name_1/log/errorcorr_reads.time
    wildcards: sample=sample_name_1
    threads: 12

Thanks in advance, Stephan

Hi Stephan,

Please send me the following log files so I can help you figure out what was the problem: /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_1.log /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.log

Vitor

Hi,

kaiju_db_custom_2.log Is empty

kaiju_db_custom_2.log

# infilename= /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa
# outfilename= /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db
# Alphabet= ACDEFGHIKLMNPQRSTVWY
# nThreads= 12
# length= 0.000000
# checkpoint= 5
# caseSens=OFF
# revComp=OFF
# term= *
# revsort=OFF
# help=OFF
readFasta: No sequences read

kaiju_db_custom_4.log (older than kaiju_db_custom_2.log)

/home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_4.log                                                                                                        801/801               100%
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed

0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
0 38.5M    0 43440    0     0  22532      0  0:29:53  0:00:01  0:29:52 22531
19 38.5M   19 7643k    0     0  2639k      0  0:00:14  0:00:02  0:00:12 2639k
49 38.5M   49 19.0M    0     0  5072k      0  0:00:07  0:00:03  0:00:04 5072k
67 38.5M   67 26.1M    0     0  5546k      0  0:00:07  0:00:04  0:00:03 5545k
87 38.5M   87 33.7M    0     0  5912k      0  0:00:06  0:00:05  0:00:01 7039k
100 38.5M  100 38.5M    0     0  6044k      0  0:00:06  0:00:06 --:--:-- 8568k
nodes.dmp

Best, Stephan

Hi Stephan,

Looks like there's still a problem with the location of the files.

Kaiju database will first convert those files (*.gbff from the path you set on the yaml file for kaiju custom database) to the file kaiju_db.faa (kaiju_db_custom_1) and then the database kaiju_db.bwt and kaiju_db.sa (kaiju_db_custom_2) and finally the index kaiju_db.fmi (kaiju_db_custom_3).

It looks like the first step is not working and the kaiju_db.faa is empty. Do you have one or more .gbff (not gzipped) files in the directory custom_fungi_viral_db/kaiju/? Notice that they are different from the files for the other tools (.fna).

If yes, can you check if the file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa is really empty?

You can get an example of such files and custom database creation in the sample data provided with the tool.

Vitor

Hi,

custom_fungi_viral_db/kaiju

~/work/epityp/mirnaseq/05_metameta/databases/custom_fungi_viral_db/kaiju$ ls -l
total 7914969
-rw-rw-r-- 1 stephan stephan 13240120830 Sep 12 09:47 fungi_viral_genomes.gbff

new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa

~/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db$ ls -l
total 23238
-rw-rw-r-- 1 stephan stephan         0 Sep 13 08:32 kaiju_db.faa
-rw-r--r-- 1 stephan stephan 111923416 Sep 13 08:32 nodes.dmp

Thanks, Stephan

Hi Stephan,

I've been trying to re-create your error and I couldn't, it runs fine in my configurations. However I noticed that the bioconda version of kaiju 1.0 installation is missing some perl runtime libraries.

Details aside, you can try to replace your file /home/stephan/work/miniconda/data/envs/py35/opt/metameta/envs/kaiju.yaml for this one kaiju.yaml. That may solve the problem.

Best, Vitor

Hi,

I tried to running the pipeline with the new kaiju.yaml file, but I'm still getting an error:

Error in job kaiju_db_custom_2 while creating output files /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.bwt, /home/stephan/work/epityp/mirnaseq/05_metam
eta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.sa.
RuleException:
CalledProcessError in line 17 of /home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm:
Command 'mkbwt -n 5 -a ACDEFGHIKLMNPQRSTVWY -nThreads 12 -o /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db /home/stephan/work/epityp/mirnaseq/05_metameta/
databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa > /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/log/kaiju_db_custom_2.log 2>&1' returned non-zero exit status 5
  File "/home/stephan/work/miniconda/data/envs/py35/opt/metameta/tools/kaiju_db_custom.sm", line 17, in __rule_kaiju_db_custom_2
  File "/home/stephan/work/miniconda/data/envs/py35/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Job failed, going on with independent jobs.
.[33mUnable to set utime on symlink sample_name_1/reads/pre1.1.fq.gz. Your Python build does not support it..[0m
Unable to set utime on symlink sample_name_1/reads/pre1.1.fq.gz. Your Python build does not support it.
1 of 19 steps (5%) done
rule errorcorr_reads:
    input: sample_name_1/reads/pre1.1.fq.gz
    output: sample_name_1/reads/pre2.1.fq.gz
    log: sample_name_1/log/errorcorr_reads.log
    benchmark: sample_name_1/log/errorcorr_reads.time
    wildcards: sample=sample_name_1
    threads: 12

.[33mUnable to set utime on symlink sample_name_1/reads/pre2.1.fq.gz. Your Python build does not support it..[0m
Unable to set utime on symlink sample_name_1/reads/pre2.1.fq.gz. Your Python build does not support it.
2 of 19 steps (11%) done
rule subsample_reads:
    input: sample_name_1/reads/pre2.1.fq.gz
    output: sample_name_1/reads/kraken.1.fq, sample_name_1/reads/motus.1.fq, sample_name_1/reads/gottcha.1.fq, sample_name_1/reads/kaiju.1.fq, sample_name_1/reads/clark.1.fq
    log: sample_name_1/log/subsample_reads.log
    benchmark: sample_name_1/log/subsample_reads.time
    wildcards: sample=sample_name_1

.[33mUnable to set utime on symlink sample_name_1/reads/kraken.1.fq. Your Python build does not support it..[0m
.[33mUnable to set utime on symlink sample_name_1/reads/gottcha.1.fq. Your Python build does not support it..[0m
.[33mUnable to set utime on symlink sample_name_1/reads/motus.1.fq. Your Python build does not support it..[0m
.[33mUnable to set utime on symlink sample_name_1/reads/kaiju.1.fq. Your Python build does not support it..[0m
.[33mUnable to set utime on symlink sample_name_1/reads/clark.1.fq. Your Python build does not support it..[0m
Unable to set utime on symlink sample_name_1/reads/kraken.1.fq. Your Python build does not support it.
Unable to set utime on symlink sample_name_1/reads/motus.1.fq. Your Python build does not support it.
Unable to set utime on symlink sample_name_1/reads/gottcha.1.fq. Your Python build does not support it.
Unable to set utime on symlink sample_name_1/reads/kaiju.1.fq. Your Python build does not support it.
Unable to set utime on symlink sample_name_1/reads/clark.1.fq. Your Python build does not support it.
3 of 19 steps (16%) done

Do you have any other ideas?

Thanks, Stephan

Hi Stephen,

Did you delete the file /home/stephan/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa before running the pipeline with the updated kaiju.yaml file?

The error is happening in a very simple step, where kaiju converts the .gbff to .faa and update the headers for further steps (with the script gbk2faa.pl). You can download this script to check if it works as an standalone, which would imply some other error in the pipeline: ./gbk2faa.pl ~/work/epityp/mirnaseq/05_metameta/databases/custom_fungi_viral_db/kaiju/fungi_viral_genomes.gbff ~/work/epityp/mirnaseq/05_metameta/databases/new_custom_fungi_viral_db/kaiju_db/kaiju_db.faa This code is exactaly what MetaMeta is failing to execute.

Besides that your .gbff file can be corrupted. But if you can open and see its content that's not the case.

Best, Vitor

Hi,

thanks for the comment - it is currently building the .faa file.

Another question:

can I use the custom database and the standard database together?

Thanks, Stephan

Hi Stephan,

Is it working because you deleted the kaiju_db.faa file or are you running the command line provided?

Yes, you can use them together by adding it to the configuration file. For example:

databases:
  - "new_custom_fungi_viral_db"
  - "archaea_bacteria"

That will run your samples for both databases (the pre-configured and the custom), generating separated results for each one of them. There is no way to run them together as one, since the tools do not provide such functionality.

Best, Vitor

Hi Vitor,

it's running because I deleted the jaiju_db.faa file.

Thanks for the information about running with multiple databases.

Best, Stephan

Hi Stephan,

Nice, so the solution is the updated version of kaiju because of the perl dependency on installation. I will fix that on the next release.

Best, Vitor

pirovc / metameta

Custom database does not work #5

Database output directory (Tip: create this folder in a common directory so it could be used for other runs as well as other users)

Sample (name and files)

Add more samples here

Custom database

Configured tools (p=profiling, b=binning) from tools folder (tool.sm and tool_db.sm)

"dudes": "p"

MetaMeta Pipeline

Number of threads for each tool (distributed among the number of cores defined by main parameter --cores)

Gzipped input files (0: not gzipped / 1: gzipped). Default: 0

Keep intermediate files (database, reads and output) (0: do not keep files / 1: keep all files). Default: 0