Closed AlessioMilanese closed 5 years ago
Thanks for the excellent report, @AlessioMilanese
I managed to reproduce it locally with v0.9.1, but it works correctly on master. Still looking into it, but might just be worthwhile to do a new release.
Could this possibly be a permission error with an erroneous error message? Does your user have permissions to create the output files inside the output directory?
I have writing permission. I can reproduce the error also with just one lane:
CCIS12370844ST-4-0_11s002754-1-1_lane5.screened.adapter.screened.hg19.pair.1.fq.gz
CCIS12370844ST-4-0_11s002754-1-1_lane5.screened.adapter.screened.hg19.pair.2.fq.gz
and also if I rename the samples to:
a_1.fastq.gz
a_2.fastq.gz
The error says
/scratch/milanese/NGless_temp_dir/reads_block_selected_mapped_reference7827-4..1.fq7827-5.gz: renameFile:renamePath:rename: does not exist (No such file or directory)
but when I check reads_block_selected_mapped_reference7827-4..1.fq7827-5.gz
before the script finish, I can see it:
-bash-4.2$ ls -lah /scratch/milanese/NGless_temp_dir/
total 648M
drwxr-xr-x. 2 milanese zeller 8 Oct 28 22:37 .
drwxr-xr-x. 16 milanese zeller 18 Oct 26 14:14 ..
-rw-r--r--. 1 milanese zeller 142M Oct 28 22:37 block_selected_mapped_reference7827-4.sam
-rw-r--r--. 1 milanese zeller 349M Oct 28 22:37 mapped_reference.7827-3.sam
-rw-r--r--. 1 milanese zeller 46M Oct 28 22:34 preprocessed.1...fq7827-0.gz
-rw-r--r--. 1 milanese zeller 46M Oct 28 22:34 preprocessed.2...fq7827-1.gz
-rw-r--r--. 1 milanese zeller 2.6M Oct 28 22:34 preprocessed.singles...fq7827-2.gz
-rw-r--r--. 1 milanese zeller 22M Oct 28 22:37 reads_block_selected_mapped_reference7827-4..1.fq7827-5.gz
-rw-r--r--. 1 milanese zeller 22M Oct 28 22:37 reads_block_selected_mapped_reference7827-4..2.fq7827-6.gz
-rw-r--r--. 1 milanese zeller 0 Oct 28 22:37 reads_block_selected_mapped_reference7827-4..singles.fq7827-7.gz
It seems indeed a problem when moving the files to the final destination.
If I change the name of the directory from CCIS12370844ST-4-0
to aa
and the file list to:
aa
it seems to work. Can you reproduce it?
Okay, I figured this out. It's a mix of (1) what I think is a bug in your code (2) NGLess does not correctly check if the output directory exists and (3) outputs a horrible error message, (4) partially because of another bug in the haskell posix library (https://github.com/haskell/unix/issues/60)
You were doing:
samples = readlines(ARGV[2])
sample = lock1(samples)
input = load_mocat_sample(ARGV[1] + '/' + sample)
You load the data from ARGV[1] </> sample
, but later write to another directory:
write(as_reads(mapped), ofile=sample+'/'+sample+'.filtered.fq.gz')
Just sample
.
If this directory does not exist, you get the stupid behaviour you saw.
I am re-classifying this bug as "better-error-message"
Hi, I'm trying to filter reads based on quality and remove human reads, but I have the following error:
Some more information: I installed ngless from bioconda (
ngless v0.9.1 (release date: July 17 2018)
) I'm running this script (/scratch/milanese/TEST/QC/filter-human-merged.ngl
):submitting it to slurm with the script (
/scratch/milanese/TEST/QC/slurm_ex2.sh
):The file
/scratch/milanese/TEST/QC/list2
contains:The directory
/g/scb2/zeller/SHARED/DATA/metaG/REAL/CRC-META/FR-CRC_N156/CCIS12370844ST-4-0
contains:The output that I get using
--trace
is: