pirovc / metameta

Other
23 stars 10 forks source link

Unable to set utime on symlink K1T0-B/reads/pre1.1.fq.gz. Your Python build does not support it. #3

Closed Thomieh73 closed 7 years ago

Thomieh73 commented 7 years ago

Hi Vitor,

I run into a few issues with metameta, when trying it out on my data. During the metameta run, I get the message, repeatedly: Unable to set utime on symlink K1T0-B/reads/pre1.1.fq.gz. Your Python build does not support it.

It seems that this is happening with all the files in the reads folder. Should I ignore this or not? I thought the miniconda came with it's own python.

The background. I have installed metameta on my area of the Abel computing cluster which uses slurm.

Installed miniconda in this way:

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
 chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh

I then made sure than no python2 / perlmodules were in my PATH

Next I added miniconda to the PATH.

export PATH=/usit/abel/u1/thhaverk/miniconda3/bin:$PATH

And then I installed metameta with the command:

conda create -c bioconda -n metameta metameta=1.1 bowtie2=2.3.0 clark=1.2.3 dudes=0.07 gottcha=1.0 jellyfish=1.1.11 kaiju=1.0 kraken=0.10.5beta krona=2.7 metametamerge=1.1 motus=1.0 spades=3.9.0 trimmomatic=0.36

With that step, I could start the metameta environment and I could run it.

I created a slurm script to start the metameta analysis, where I call metameta with the command:

metameta --configfile MM_config_3.yaml --keep-going -j 999 --cluster-config MM_cluster.json --cluster "sbatch --account=nn9244k --mem-per-cpu={cluster.mem} --partition={cluster.partition} --cpus-per-task={cluster.cpus-per-task} --time={cluster.time} --job-name={cluster.job-name} --output={cluster.output}"

The first bit of output from metameta:

Starting job 18309009 ("MetaMeta") on c16-12 at Mon Sep 11 10:37:24 CEST 2017 setting-up metameta environment /work/users/thhaverk/scripts activating metameta environment Provided cluster nodes: 999 Job counts: count jobs 1 all 1 clark_rpt 1 clark_run_1 6 clean_files 1 clean_reads 1 dudes_rpt 1 dudes_run_1 1 dudes_run_2 1 errorcorr_reads 1 gottcha_rpt 1 gottcha_run_1 1 kaiju_rpt 1 kaiju_run_1 1 kraken_rpt 1 kraken_run_1 1 krona 1 metametamerge 1 motus_rpt 1 motus_run_1 1 subsample_reads 1 trim_reads 26 rule trim_reads: input: /projects/cees/in_progress/anthrax/data/clean_data/K1T0-B/K1T0-B.pair_1.fastq.gz output: K1T0-B/reads/pre1.1.fq.gz log: K1T0-B/log/trim_reads.log benchmark: K1T0-B/log/trim_reads.time wildcards: sample=K1T0-B threads: 16

Unable to set utime on symlink K1T0-B/reads/pre1.1.fq.gz. Your Python build does not support it. 1 of 26 steps (4%) done rule errorcorr_reads: input: K1T0-B/reads/pre1.1.fq.gz output: K1T0-B/reads/pre2.1.fq.gz log: K1T0-B/log/errorcorr_reads.log benchmark: K1T0-B/log/errorcorr_reads.time wildcards: sample=K1T0-B threads: 16

Unable to set utime on symlink K1T0-B/reads/pre2.1.fq.gz. Your Python build does not support it. 2 of 26 steps (8%) done

pirovc commented 7 years ago

Hi Thomas,

This is a known warning. It happens because I'm using a symbolic link system while handling the reads to save disk space and there's some incompability with snakemake setting the timestamp on them. However the pipeline should work just fine even with the warnings, so you can ignore them. I wish I solve that for a next relase.

Best, Vitor

Thomieh73 commented 7 years ago

Okay, thanks for the quick response. I will ignore it then.