No such file or directory

[Caro-Ca]

Hello, I have got ~2000 files of paired-end reads. I divided them into four folders in order to run McClintock on different CPUs. I could successfully run on two folders, but not on the last two folders giving me this error:

silviav@wildtype1:/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000:for i in *paired_R1.fastq ; do /home/silviav/mcclintock/mcclintock.sh -r /home/silviav/mcclintock/test/sacCer2.fasta -c /home/silviav/mcclintock/test/sac_cer_TE_seqs.fasta -g /home/silviav/mcclintock/test/reference_TE_locations.gff -t /home/silviav/mcclintock/test/sac_cer_te_families.tsv -1 $i -2 "${i:0:19}2.fastq" -p 36; done

Running McClintock version: 
Running McClintock version: 1d06a94bf7bcd5a1e17dfc8336fb82895780c713
1d06a94bf7bcd5a1e17dfc8336fb82895780c713

Date of run is 22_03_20

Date of run is 22_03_20

Creating directory structure...

Creating directory structure...

cp: cannot stat 'ERR1309028_paired_R1.fastq': No such file or directory
cp: cannot stat 'ERR1309028_paired_R2.fastq': No such file or directory

Performing FastQC analysis...

Performing FastQC analysis...

Skipping '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/sacCer2/ERR1309028_paired_R1/reads/ERR1309028_paired_R1.fastq' which didn't exist, or couldn't be read
Skipping '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/sacCer2/ERR1309028_paired_R1/reads/ERR1309028_paired_R2.fastq' which didn't exist, or couldn't be read
[bwa_index] Pack FASTA... 0.10 sec
[bwa_index] Construct BWT for the packed sequence...
^X^C
silviav@wildtype1:/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000:ls -ltrhs ERR1309028_paired_R1.fastq
5.8G -rw-r--r-- 1 silviav binf 5.8G Dec 14 10:01 ERR1309028_paired_R1.fastq
silviav@wildtype1:/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000:ls -ltrhs ERR1309028_paired_R2.fastq
5.8G -rw-r--r-- 1 silviav binf 5.8G Dec 14 10:01 ERR1309028_paired_R2.fastq

I also tried running on absolute paths but again I got an error.

silviav@wildtype1:/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000:for i in /hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/*paired_R1.fastq ; do /home/silviav/mcclintock/mcclintock.sh -r /home/silviav/mcclintock/test/sacCer2.fasta -c /home/silviav/mcclintock/test/sac_cer_TE_seqs.fasta -g /home/silviav/mcclintock/test/reference_TE_locations.gff -t /home/silviav/mcclintock/test/sac_cer_te_families.tsv -1 /hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/from701_1000/$i -2 /hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/from701_1000/"${i:0:19}2.fastq" -p 36; done

Running McClintock version: 
Running McClintock version: 1d06a94bf7bcd5a1e17dfc8336fb82895780c713
1d06a94bf7bcd5a1e17dfc8336fb82895780c713

Date of run is 22_03_20

Date of run is 22_03_20

Creating directory structure...

Creating directory structure...

cp: cannot stat '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/from701_1000//hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/ERR1309028_paired_R1.fastq': No such file or directory
cp: cannot stat '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/from701_1000//hosts/linuxhome/ch2.fastq': No such file or directory

Performing FastQC analysis...

Performing FastQC analysis...

Skipping '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/sacCer2/ERR1309028_paired_R1/reads/ERR1309028_paired_R1.fastq' which didn't exist, or couldn't be read
Skipping '/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000/sacCer2/ERR1309028_paired_R1/reads/ch2.fastq' which didn't exist, or couldn't be read
[bwa_index] Pack FASTA... 0.10 sec
[bwa_index] Construct BWT for the packed sequence...
^X^C

Additionally, miniconda is activated:

silviav@wildtype1:/hosts/linuxhome/chaperone/silviav/reads/Peter/paired_IMPORTANTE/From701_1000:which activate 
activate is /home/silviav/miniconda/bin/activate
activate is /home/silviav/anaconda2/bin/activate

Thank you in advance for your help.

[cbergman]

• Have you tried running McClintock directly on the paired end data for ERR1309028 (i.e. not inside a loop)?
• Also the errors for the relative and absolute path invocations are different. Perhaps this can give you some insight into your problem.
• Also, can you please email me using the address on this page: https://www.genetics.uga.edu/directory/casey-bergman. It would be helpful to have a chat off of github to understand what you are trying to do.

[Caro-Ca]

Problem solved. I really had to specify the absolute paths in everything. Sorry for any inconvenience. Thank you for your time and help.

[cbergman]

Hi

This is the third or fourth situation where we’ve been providing support for general Unix/python issues. As I said previously, please email to discuss your project if you would like general bioinformatics support. Otherwise we will direct you to post general bioinformatics questions at biostars.

Thanks!