search

CSB5 / OPERA-MS

OPERA-MS - Hybrid Metagenomic Assembler
Other
88 stars 17 forks source link

Error in the long read processing #79

Open ArtyomVodnev opened 1 year ago

ArtyomVodnev commented 1 year ago

Hello, I'm trying to run OPERA-MS using this command: perl /home/arvodnev/OPERA-MS/OPERA-MS.pl --contig-file /home/arvodnev/test_project/AssembledSR/SPAdes/contigs.fasta --short-read1 /home/arvodnev/test_project/rawdata/SR/trimmed_1.fastq --short-read2 /home/arvodnev/test_project/rawdata/SR/trimmed_2.fastq --long-read /home/arvodnev/test_project/rawdata/LR/SRR23215007.fastq --out-dir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY --long-read-mapper minimap2

And it stop running with error: `arvodnev@darwin ~/test_project $ ./HY.sh

mkdir -p /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY

version v0.9.0

[Thu Apr 20 11:37:04 2023] Short read assembly [1/9] [Thu Apr 20 11:37:04 2023] Skip [contig file provided as input]

rm -r /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly

mkdir -p /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly

[Thu Apr 20 11:37:06 2023] Long-read mapping and assembly graph generation [2/9]

rm -r /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping

mkdir -p /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping

/home/arvodnev/OPERA-MS//tools_opera_ms/perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//OPERA-long-read.pl --contig-file /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta --kmer 60 --long-read-file /home/arvodnev/test_project/rawdata/LR/SRR23215007.fastq --output-prefix opera --output-directory /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping --num-of-processors 2 --opera /home/arvodnev/OPERA-MS//OPERA-LG/bin/ --illumina-read1 /home/arvodnev/test_project/rawdata/SR/trimmed_1.fastq --illumina-read2 /home/arvodnev/test_project/rawdata/SR/trimmed_2.fastq --samtools-dir /home/arvodnev/OPERA-MS//tools_opera_ms/ --perl-dir /home/arvodnev/OPERA-MS//tools_opera_ms/ --minimap2 /home/arvodnev/OPERA-MS//tools_opera_ms/ --short-read-tooldir /home/arvodnev/OPERA-MS//tools_opera_ms/ --skip-opera 1 --skip-short-read-mapping 1 > /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/OPERA-long-read.out 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/OPERA-long-read.err

Error in the long read processing. Please see log for details /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/OPERA-long-read.out /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/OPERA-long-read.err. ` the file content of OPERA-long-read.err is: sh: 1: /home/arvodnev/OPERA-MS//tools_opera_ms/perl: not found

the file content of OPERA-long-read.out is empty

I will be glad if you can help me find a solution to this error. Thank you!

jsgounot commented 1 year ago

Hi Artyom, most likely your perl executable (/home/arvodnev/OPERA-MS/tools_opera_ms/perl) is broken. You can try that: rm /home/arvodnev/OPERA-MS/tools_opera_ms/perl && ln -s $(which perl) /home/arvodnev/OPERA-MS/tools_opera_ms/perl.

ArtyomVodnev commented 1 year ago

Thank you! The solution you suggested really helped!

Unfortunately, I got another error after some time after starting the program: `/home/arvodnev/OPERA-MS//tools_opera_ms/perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//gapfilling.pl /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta /home/arvodnev/test_project/rawdata/LR/SRR23215007.fastq /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping 2 499 /home/arvodnev/OPERA-MS//OPERA-LG/bin/ /home/arvodnev/OPERA-MS//tools_opera_ms/ /home/arvodnev/OPERA-MS//tools_opera_ms/ /home/arvodnev/OPERA-MS//tools_opera_ms/ /home/arvodnev/OPERA-MS//tools_opera_ms//MUMmer3.23/ 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/gap_filling.err

Error in during gap_filling. Please see /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/gap_filling.err for details.`

The file content of gap_filling.err is: `/home/arvodnev/OPERA-MS//tools_opera_ms//perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//extract_read_sequence.pl --edge-file /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/edge_read_info.dat --contig-file /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta --opera-lr-dir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/../ --read-file /home/arvodnev/test_project/rawdata/LR/SRR23215007.fastq --output-directory /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/extract_read.err

cat /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/consensus_cmd.sh | xargs -L 1 -P 2 -I COMMAND sh -c "COMMAND" 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/consensus_cmd.sh.log

mkdir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING

rm /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa; for f in /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/operascaffold/racon.fa; do sed 's/Consensus_//' $f >> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa; done rm: cannot remove '/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa': No such file or directory

/home/arvodnev/OPERA-MS//tools_opera_ms//perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//run_mummer_large_ref.pl /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling.coords 2 /home/arvodnev/OPERA-MS//tools_opera_ms//MUMmer3.23/ > /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.out 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.err

Error during tilling generation. Please see /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.out and /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.err for details.`

The file content of tilling_1.err is: `rm -r /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF rm: cannot remove '/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY': No such file or directory rm: cannot remove '/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF': No such file or directory mkdir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF split /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa mkdir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY split /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta *** run the mummer mapping rm -r /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer rm: cannot remove '/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer': No such file or directory mkdir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer cat /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer/cmd_delta.txt | xargs -I cmd --max-procs=2 bash -c cmd > /dev/null

Error in during nucmer.`

The file content of tilling_1.out is empty.

I would be very grateful if you can advise what should be done to fix this error. Thank you again!

jsgounot commented 1 year ago

Hi,

your issue seems similar to #60 - Could you run one command line from the cmd_delta.txt as mentioned in this thread?

ArtyomVodnev commented 1 year ago

I run one command line from the cmd_delta.txt, but I got nothing The command line I run is: /home/arvodnev/OPERA-MS//tools_opera_ms//MUMmer3.23//nucmer --nosimplify --maxmatch -p /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer/split_3.fa_split_1.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF/split_3.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY/split_1.fa > /dev/null 2> /dev/null

In addition, the mummer directory I found in GAPFILLING, and not in TILLING directory /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer

jsgounot commented 1 year ago

You have to remove the redirection in the command line: /home/arvodnev/OPERA-MS//tools_opera_ms//MUMmer3.23//nucmer --nosimplify --maxmatch -p /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/mummer/split_3.fa_split_1.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF/split_3.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY/split_1.fa

ArtyomVodnev commented 1 year ago

After removing the redirection, this is what I got: ERROR: The following critical files could not be used /home/arvodnev/test_project/OPERA-MS/tools_opera_ms/MUMmer3.23/aux_bin/postnuc /home/arvodnev/test_project/OPERA-MS/tools_opera_ms/MUMmer3.23/aux_bin/prenuc /home/arvodnev/test_project/OPERA-MS/tools_opera_ms/MUMmer3.23/mgaps /home/arvodnev/test_project/OPERA-MS/tools_opera_ms/MUMmer3.23/mummer Check your paths and file permissions and try again

jsgounot commented 1 year ago

Did you move your OPERA-MS folder after installation? Anyway, you need to have a working nucmer for OPERA-MS to proceed. Maybe try to reinstall with your new location.

ArtyomVodnev commented 1 year ago

Yes, I moved it to the previous directory, after installation. That is, I need to uninstall and reinstall the program?

jsgounot commented 1 year ago

Hard to know for sure but that's definitely what you should try first.

ArtyomVodnev commented 1 year ago

Okay, thank you! I'll try and let you know if it helps.

ArtyomVodnev commented 1 year ago

Hello, Unfortunately, after I reinstalled the program, I got the error again: Error in during gap_filling. Please see /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/gap_filling.err for details.

the file content of gap_filling.err is:

/home/arvodnev/OPERA-MS//tools_opera_ms//perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//extract_read_sequence.pl --edge-file /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/read_mapping/edge_read_info.dat --contig-file /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta --opera-lr-dir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/../ --read-file /home/arvodnev/test_project/rawdata/LR/SRR23215007.fastq --output-directory /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/extract_read.err

cat /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/consensus_cmd.sh | xargs -L 1 -P 2 -I COMMAND sh -c "COMMAND" 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/consensus_cmd.sh.log

mkdir /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING

rm /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa; for f in /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/*opera_scaffold_*/racon.fa; do sed 's/Consensus_//' $f >> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa; done
rm: cannot remove '/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa': No such file or directory

/home/arvodnev/OPERA-MS//tools_opera_ms//perl /home/arvodnev/OPERA-MS//OPERA-LG/bin//run_mummer_large_ref.pl /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/REF /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/QUERY /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling.coords 2 /home/arvodnev/OPERA-MS//tools_opera_ms//MUMmer3.23/ > /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.out 2> /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1.err

/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/..//scaffolds.scaf
/home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling.coords
Gap length: 932742
Tiling length: 110761472     116456783
Fraction gap 0.00835085333963673
 *** get_contig_and_corrected_read_in_gap /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling.paf  /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/user_assembly/contigs.fasta  /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling_contig.fa
python /home/arvodnev/OPERA-MS//OPERA-LG/bin//Remap.py /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling.paf /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/consensus.fa /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/first_tilling_contig.fa > /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1_remap.out
sh: 1: python: not found

Error during contig remapping. Please see /home/arvodnev/test_project/AssemblyHY/OPERA-MS_HY/intermediate_files/opera_long_read/GAPFILLING/TILLING/tilling_1_remap.out for details.

the file content of tilling_1_remap.out is empty.

Is there any other way to fix this?

jsgounot commented 1 year ago

Hi, it clearly says python is missing. Please try to resolve all your basic installation problems first.