search

alekseyzimin / masurca

GNU General Public License v3.0
243 stars 35 forks source link

RuntimeError Usage: Options_minmatch(self,m) #38

Open MDSharma opened 6 years ago

MDSharma commented 6 years ago

Hello,

We seem to be getting "RuntimeError Usage: Options_minmatch(self,m)" errors in our MaSurca run. Please see brief logs below:

[Fri Jun  1 09:54:31 BST 2018] Estimated genome size: 536282973
[Fri Jun  1 09:54:31 BST 2018] Creating k-unitigs with k=41
[Fri Jun  1 10:06:19 BST 2018] Creating k-unitigs with k=31
[Fri Jun  1 10:25:34 BST 2018] Filtering mate pairs
..
..
Output prefix mr.41.15.17.0.029
Pacbio coverage >10x, using 10x of the longest reads
Reducing super-read k-mer size
Mega-reads pass 1
Running locally in 1 batch
compute_psa 6939941 1423963606
Processed 500000 super reads, irreducible 420469, processing 14285 super reads per second
Processed 1000000 super reads, irreducible 769474, processing 35714 super reads per second
..
..
Processed 3500000 super reads, irreducible 2410744, processing 38461 super reads per second
Processed 4000000 super reads, irreducible 2783819, processing 45454 super reads per second
Mega-reads pass 2
Running locally in 1 batch
compute_psa 3073547 2330298699
Refining alignments
Attempt to free unreferenced scalar: SV 0x32db870, Perl interpreter: 0x7d5010 at /cm/shared/apps/MaSuRCA/3.2.6/bin/../lib/perl/mummer.pm line 262, <STDIN> line 2002.
..
..

Attempt to free unreferenced scalar: SV 0x558fd80, Perl interpreter: 0x7d5010 at /cm/shared/apps/MaSuRCA/3.2.6/bin/../lib/perl/mummer.pm line 262, <STDIN> line 18346.
RuntimeError Usage: Options_minmatch(self,m); at /cm/shared/apps/MaSuRCA/3.2.6/bin/refine_alignments.pl line 51, <STDIN> line 18346.
Joining
refine/join alignments failed
[Sat Jun  2 00:20:26 BST 2018] Assembly stopped or failed, see CA.mr.41.15.17.0.029.log

Any thoughts on whats going on here? Our first guess was that the error is coming from mummer where the --minmatch limit is causing issues.. The log file referenced at the assembly failure doe snot seem to exist at all so we could not debug further.

MDSharma commented 6 years ago

Hi Folks, we amended the runtime options in refine_alignments.pl for the mummer step to use the default minmatch of 20 (instead of the 10 thats hardcoded into the script) but the process still failed as below:

$ tail -f masurca_run.log
[Fri Jul  6 08:52:27 BST 2018] Processing sj library reads
[Fri Jul  6 08:56:42 BST 2018] Average PE read length 82
[Fri Jul  6 08:56:42 BST 2018] Using kmer size of 41 for the graph
[Fri Jul  6 08:56:43 BST 2018] MIN_Q_CHAR: 33
[Fri Jul  6 08:56:43 BST 2018] Creating mer database for Quorum
[Fri Jul  6 09:01:37 BST 2018] Error correct PE.
[Fri Jul  6 09:06:45 BST 2018] Error correct JUMP.
[Fri Jul  6 09:10:09 BST 2018] Estimated genome size: 536282973
[Fri Jul  6 09:10:09 BST 2018] Creating k-unitigs with k=41
[Fri Jul  6 09:18:28 BST 2018] Creating k-unitigs with k=31
[Fri Jul  6 09:29:48 BST 2018] Filtering mate pairs
Assuming outtie orientation
Assuming outtie orientation
Chimeric/Redundant jump reads:
 42405398 chimeric_sj.txt
 32066276 redundant_sj.txt
 74471674 total
[Fri Jul  6 11:08:11 BST 2018] Creating FRG files
[Fri Jul  6 11:13:36 BST 2018] Computing super reads from PE
[Fri Jul  6 12:21:32 BST 2018] Using linking mates
Using CABOG from is /cm/shared/apps/MaSuRCA/3.2.6/bin/../CA8/Linux-amd64/bin
Running mega-reads correction/assembly
Using mer size 15 for mapping, B=17, d=0.029
Estimated Genome Size 536282973
Estimated Ploidy 1
Using 120 threads
Output prefix mr.41.15.17.0.029
Pacbio coverage >10x, using 10x of the longest reads
Joining
refine/join alignments failed
[Fri Jul  6 12:21:44 BST 2018] Assembly stopped or failed, see CA.mr.41.15.17.0.029.log

CA.mr.41.15.17.0.029.log still does not exist!!!

Please could you shed some light on this..

Clearly changing the minmatch to mummer default made the steps go further but the CA.mr* log file does not get created when this step fails so we don't know where else to look

MDSharma commented 6 years ago

We have repeated this analysis with v3.2.4 and 3.2.6 and the outcome remains unaltered. Trying to compile an older / earlier version as well. @alekseyzimin your views will be much appreciated.

SvitlanaLukicheva commented 5 years ago

Hello,

I got the same error with v3.2.8.

Here is the output of the program:

jellyfish OK
runCA OK
createSuperReadsForDirectory.perl OK
creating script file for the actions...done.
execute assemble.sh to run assembly
[Wed Nov 21 15:35:06 CET 2018] Processing pe library reads
[Wed Nov 21 15:57:32 CET 2018] Average PE read length 250
[Wed Nov 21 15:57:32 CET 2018] Using kmer size of 83 for the graph
[Wed Nov 21 15:57:32 CET 2018] MIN_Q_CHAR: 33
[Wed Nov 21 15:57:32 CET 2018] Creating mer database for Quorum
[Wed Nov 21 16:36:08 CET 2018] Error correct PE
[Wed Nov 21 20:50:52 CET 2018] Estimating genome size
[Wed Nov 21 21:31:14 CET 2018] Estimated genome size: 1525394691
[Wed Nov 21 21:31:14 CET 2018] Creating k-unitigs with k=83
[Thu Nov 22 00:53:49 CET 2018] Computing super reads from PE
[Thu Nov 22 07:49:38 CET 2018] Using CABOG from /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin
[Thu Nov 22 07:49:38 CET 2018] Running mega-reads correction/assembly
[Thu Nov 22 07:49:38 CET 2018] Using mer size 15 for mapping, B=15, d=0.02
[Thu Nov 22 07:49:38 CET 2018] Estimated Genome Size 1525394691
[Thu Nov 22 07:49:38 CET 2018] Estimated Ploidy 1
[Thu Nov 22 07:49:38 CET 2018] Using 16 threads
[Thu Nov 22 07:49:38 CET 2018] Output prefix mr.41.15.15.0.02
[Thu Nov 22 07:49:38 CET 2018] Using 25x of the longest ONT reads
[Thu Nov 22 07:49:38 CET 2018] Reducing super-read k-mer size
[Thu Nov 22 09:17:48 CET 2018] Mega-reads pass 1
[Thu Nov 22 09:17:48 CET 2018] Running locally in 1 batch
[Thu Nov 22 10:14:27 CET 2018] Mega-reads pass 2
[Thu Nov 22 10:14:27 CET 2018] Running locally in 1 batch
[Thu Nov 22 10:17:37 CET 2018] Refining alignments
[Thu Nov 22 10:17:38 CET 2018] Joining
[Thu Nov 22 10:17:38 CET 2018] Gap consensus
[Thu Nov 22 10:17:38 CET 2018] Warning! Some or all gap consensus jobs failed, see files in mr.41.15.15.0.02.join_consensus.tmp, proceeding anyway, to rerun gap consensus erase mr.41.15.15.0.02.1.fa and re-run assemble.sh
[Thu Nov 22 10:17:38 CET 2018] Assembly stopped or failed, see CA.mr.41.15.15.0.02.log

And below is the output of the error file:

RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, <STDIN> line 298.
Attempt to free unreferenced scalar: SV 0x18b0978, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, <STDIN> line 2002.
RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, <STDIN> line 2002.
Attempt to free unreferenced scalar: SV 0x2708240, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, <STDIN> line 2004.
RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, <STDIN> line 2004.
Attempt to free unreferenced scalar: SV 0x45be6a0, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, <STDIN> line 3744.
RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, <STDIN> line 3744.
xargs: invalid number for -P option
Usage: xargs [OPTION]... COMMAND INITIAL-ARGS...
Run COMMAND with arguments INITIAL-ARGS and more arguments read from input.

The CA.mr* log is not created.

I already successfully executed MaSuRCA v2.3.5 with a similar configuration and a similar data (but a lower coverage of the ONT data, so I changed the value of USE_LINKING_MATES to 0 this time). Could it be related to this parameter?

alekseyzimin commented 5 years ago

Hi,

No, it is not related to linking mates. Try re-running install.sh and capturing output and look for any errors.

--Aleksey

On Thu, Nov 22, 2018 at 8:31 AM Svitlana Lukicheva notifications@github.com wrote:

Hello,

I got the same error with v3.2.8.

Here is the output of the program:

jellyfish OK runCA OK createSuperReadsForDirectory.perl OK creating script file for the actions...done. execute assemble.sh to run assembly [Wed Nov 21 15:35:06 CET 2018] Processing pe library reads [Wed Nov 21 15:57:32 CET 2018] Average PE read length 250 [Wed Nov 21 15:57:32 CET 2018] Using kmer size of 83 for the graph [Wed Nov 21 15:57:32 CET 2018] MIN_Q_CHAR: 33 [Wed Nov 21 15:57:32 CET 2018] Creating mer database for Quorum [Wed Nov 21 16:36:08 CET 2018] Error correct PE [Wed Nov 21 20:50:52 CET 2018] Estimating genome size [Wed Nov 21 21:31:14 CET 2018] Estimated genome size: 1525394691 [Wed Nov 21 21:31:14 CET 2018] Creating k-unitigs with k=83 [Thu Nov 22 00:53:49 CET 2018] Computing super reads from PE [Thu Nov 22 07:49:38 CET 2018] Using CABOG from /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin [Thu Nov 22 07:49:38 CET 2018] Running mega-reads correction/assembly [Thu Nov 22 07:49:38 CET 2018] Using mer size 15 for mapping, B=15, d=0.02 [Thu Nov 22 07:49:38 CET 2018] Estimated Genome Size 1525394691 [Thu Nov 22 07:49:38 CET 2018] Estimated Ploidy 1 [Thu Nov 22 07:49:38 CET 2018] Using 16 threads [Thu Nov 22 07:49:38 CET 2018] Output prefix mr.41.15.15.0.02 [Thu Nov 22 07:49:38 CET 2018] Using 25x of the longest ONT reads [Thu Nov 22 07:49:38 CET 2018] Reducing super-read k-mer size [Thu Nov 22 09:17:48 CET 2018] Mega-reads pass 1 [Thu Nov 22 09:17:48 CET 2018] Running locally in 1 batch [Thu Nov 22 10:14:27 CET 2018] Mega-reads pass 2 [Thu Nov 22 10:14:27 CET 2018] Running locally in 1 batch [Thu Nov 22 10:17:37 CET 2018] Refining alignments [Thu Nov 22 10:17:38 CET 2018] Joining [Thu Nov 22 10:17:38 CET 2018] Gap consensus [Thu Nov 22 10:17:38 CET 2018] Warning! Some or all gap consensus jobs failed, see files in mr.41.15.15.0.02.join_consensus.tmp, proceeding anyway, to rerun gap consensus erase mr.41.15.15.0.02.1.fa and re-run assemble.sh [Thu Nov 22 10:17:38 CET 2018] Assembly stopped or failed, see CA.mr.41.15.15.0.02.log

And below is the output of the error file:

RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, line 298. Attempt to free unreferenced scalar: SV 0x18b0978, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, line 2002. RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, line 2002. Attempt to free unreferenced scalar: SV 0x2708240, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, line 2004. RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, line 2004. Attempt to free unreferenced scalar: SV 0x45be6a0, Perl interpreter: 0x701010 at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/../lib/perl/mummer.pm line 262, line 3744. RuntimeError Usage: Options_minmatch(self,m); at /svub0/slukiche/assemblers/masurca/MaSuRCA-3.2.8/bin/refine_alignments.pl line 51, line 3744. xargs: invalid number for -P option Usage: xargs [OPTION]... COMMAND INITIAL-ARGS... Run COMMAND with arguments INITIAL-ARGS and more arguments read from input.

The CA.mr* log is not created.

I already successfully executed MaSuRCA v2.3.5 with a similar configuration and a similar data (but a lower coverage of the ONT data, so I changed the value of USE_LINKING_MATES to 0 this time). Could it be related to this parameter?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alekseyzimin/masurca/issues/38#issuecomment-441031827, or mute the thread https://github.com/notifications/unsubscribe-auth/AZ9zHbqquok0Uh4WIH6bT0rHeXzVSCeqks5uxqcygaJpZM4VApt4 .

-- Dr. Alexey V. Zimin Associate Research Scientist Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA (301)-437-6260 http://www.genome.umd.edu http://masurca.blogspot.com

SvitlanaLukicheva commented 5 years ago

You are right, it was an installation related issue. I reinstalled MaSuRCA and now it works, thank you!

jsoghigian commented 4 years ago

My error appears to be related to this error, so posting here.

I did reinstall MaSuRCA and still received the error:

Attempt to free unreferenced scalar: SV 0x30b6ad8, Perl interpreter: 0x1546010 at /my/hidden/path/MaSuRCA-3.4.2/bin/../lib/perl/mummer.pm line 262, <STDIN> line 3110.
Attempt to free unreferenced scalar: SV 0x2584f58, Perl interpreter: 0x12c1010 at /my/hidden/path/MaSuRCA-3.4.2/bin/../lib/perl/mummer.pm line 262, <STDIN> line 2002.
RuntimeError Usage: Options_minmatch(self,m); at /my/hidden/path/MaSuRCA-3.4.2/bin/refine_alignments.pl line 51, <STDIN> line 2002.
RuntimeError Usage: Options_minmatch(self,m); at /my/hidden/path/MaSuRCA-3.4.2/bin/refine_alignments.pl line 51, <STDIN> line 3110.
jsoghigian commented 4 years ago

As I remembered previously running masurca some time ago on the cluster I was using, I loaded an earlier perl version that seemed to resolve the issue. May be an issue with the installation of Perl 5.28 on the cluster and not a masurca issue. Was able to complete my assembly without problems now.

lhui2010 commented 2 years ago

Hi,

I met the same problem as jsoghigian did. Attempt to free unreferenced scalar: SV ....

I checked installation logs and found errors during compiling samtools. After manual compiling, no installation errors were reported again. I rerun masurca, but the problem persist.

According to jsoghigian's suggestion, I changed perl v5.26.2 (conda perl) to perl v5.16.3 (The system default perl), and restart it. This problem was probably solved (still running refine_alignment step and no error reported yet). Thanks guys.

Hui