ERROR: sortBNX failed

[xiaoquexingchen]

Hi,

I use the following command "python /home/len/tools/pipeline/Solve3.6.1_11162020/Pipeline/1.0/pipelineCL.py -T 2 -j 4 -i 5 -R -w -d -U -je 4 -jp 4 -J 4 -TJ 4 -Te 4 -Tp 4 -B 0 -t /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0 -b /home/len/all.bnx -r /root/output /NA12878_DLE1_20kb_5labels.cmap -l denovo -a /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/optArguments_nonhaplotype_saphyr_human.xml" When performing denovo, the following error occurs:

"Prerun Tests: 0 ERRORS 0 WARNINGS

Hostname: localhost.localdomain

Tools Version: N/A

Solve Version: N/A

Pipeline Version: 11646

RefAligner Version: 11643

Assembly ID: 162

Pipeline start time: Sat Apr 10 22:28:44 2021

('checkScanScaling: autoNoise=', False) Not performing autoNoise (see -y): setting doScanScale = False, hence not using _rescaled.bnx for BNX splits

Executing stage number 1: AutoNoise + SplitBNX

Molecule Stats (/home/len/denovo/all.bnx): Total number of molecules: 1147524 Total length (Mbp): 268845.841 Average length (kbp): 234.283 Molecule N50 (kbp): 233.550 Label density (/100kb): 17.395

Sorting /home/len/denovo/all.bnx into /home/len/denovo/all_sorted

Starting Multi-Threaded Process: SortBNX Running 1 jobs with 4 threads, sleepTime=0.01 START 1: SortBNX, 4 Thr, 1 R, 1 T, 0 F, 0 Q STOP 1: SortBNX, 4 Thr, 0 R, 1 T, 1 F, 0 Q TotalTime = 0h 0.57m RunTime = 0h 0.57m CPUload=73% host=NA Finished Multi-Threaded Process: SortBNX

ERROR: sortBNX failed. Check: /home/len/denovo/all.bnx

WARNING: missing xmap file: /home/len/denovo/contigs/exp_refineFinal1_sv/merged_smaps/exp_refineFinal1_merged.xmap

Pipeline end time: Sat Apr 10 22:29:21 2021 Elapsed time: 0.63m; 0.01h; 0.00d

Warning/Error messages: ('warning','Missing end marker in \"/home/len/denovo/all_sorted.stdout\" (found \"33472) failed\\n\" while expecting \"END of output \")\n') ('error','job has not completed, see stdout=\"/home/len/denovo/all_sorted.stdout\"') ('critical','ERROR: sortBNX failed. Check: /home/len/denovo/all.bnx')

Warning/Error summary: 1 warning(s) 1 critical(s) 1 error(s)

Pipeline has failed"

I tried the previous schemes, but failed. I would like to ask you how to solve this problem, and I look forward to your reply.

[xiaoquexingchen]

Hi, In addition, I also encountered a problem that I used the command "./runBNG bnxfilter -b /home/len/all.bnx -t 2 -m 2 -p molecule120kb -r /home/len/tools/pipeline/Solve3.6.1 _11162020/RefAligner/1.0/RefAligner -l 120 -o /root/output "When filtering, the following problem occurred:

Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 WARNING: host localhost.localdomain does not support AMD Zen2 or Intel Skylake avx2: switching to Haswell avx2 binary execname=/home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel,q=/RefAligner,p= Trying alternate binary: /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 Appending stdout to /root/output/molecule120kb.stdout Appending stderr to /root/output/molecule120kb.stdout

Hope to get your help, thank you very much!

[yyx8671]

Hi @xiaoquexingchen,

For your first question, you may try to increase the number of total CPUs (-T). I see you used 2 CPU threads for your task, but for subjobs you used 4 threads, which might be the reason to crash the run.

For your second question, it seems your server has a different configuration and the program was trying to find the suitable RefAligner/Assembler to match with your system. If the task still failed, you may try to use the binary file under the avx folder.

By the way, for any issue related to Bionano Sovle rather than runBNG, you may contact Bionano Genomics directly for help.

Thanks, Andy

[xiaoquexingchen]

Hi, I changed the input to "./runBNG denovo -H yes -M yes -e yes -t /home/len/tools/pipeline/1.0/RefAligner/1.0 -s /home/len/tools/pipeline/Solve3.6.1 _11162020 -b /home/len/all.bnx -r /home/len/hg19_DLE1_0kb_0labels.cmap -T 16 -j 16 -i 5 -o /root/output2 -a 9 -z 3379 -B 0", but always reported with the same error, I still couldn't solve the problem. Is there a problem with my operation?

"ERROR: sortBNX failed. Check: /root/output2/all.bnx

WARNING: missing xmap file: /root/output2/contigs/exp_refineFinal1_sv/merged_smaps/exp_refineFinal1_merged.xmap

Pipeline end time: Sun Apr 11 14:40:25 2021 Elapsed time: 0.60m; 0.01h; 0.00d

Warning/Error messages: ('warning','Missing end marker in \"/root/output2/all_sorted.stdout\" (found \"33472) failed\\n\" while expecting \"END of output\ ")\n') ('error','job has not completed, see stdout=\"/root/output2/all_sorted.stdout\"') ('critical','ERROR: sortBNX failed. Check: /root/output2/all.bnx')

Warning/Error summary: 1 warning(s) 1 critical(s) 1 error(s)

Pipeline has failed"

Thanks

[xiaoquexingchen]

Hi, Does the above problem need to install the sge configuration path?

Thanks

[yyx8671]

Hi @xiaoquexingchen,

You may try the default '-B' in runBNG. Please also ensure the label density (-a) is 9 (To my knowledge, the DLE1 density would not be that low).

As I mentioned in other posts and the READMEfile, runBNG does not need any other installation except some python2 and R packages required by Bionano Solve.

[xiaoquexingchen]

Hi, I changed -a to 30 and -B to the default value, but the same error still occurred. And when I use the MQR function, when I enter "./runBNG MQR -b /home/len/all.bnx -r /home/len/hg19_DLE1_0kb_0labels.cmap -R /home/len/tools/pipeline/Solve3.6.1_11162020 /RefAligner/1.0/RefAligner -t 2 -m 2 -o /root -p MQR", there is also a problem:

"./RunBNG MQR -b /home/len/all.bnx -r /home/len/hg19_DLE1_0kb_0labels.cmap -R /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/RefAligner -t 2- m 2 -o /root -p MQR ====================================== Checking the molecule quality ======== ===============================

Start date: Sunday, April 11, 2021 15:46:01 CST

The bnx file is: /home/len/all.bnx

The ref is: /home/len/hg19_DLE1_0kb_0labels.cmap

The RefAligner is: /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/RefAligner

The maximum threads are: 2

The maximum memory (Gb) are: 2

The minimun filtered single molecule length (Kb) is: 150

Iteriations for each hashtable: 3

Times to regenerate the hashtable: 3

Cutoff of the label interval differences: 40

The output file names are: /root/MQR*

Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 WARNING: host localhost.localdomain does not support AMD Zen2 or Intel Skylake avx2: switching to Haswell avx2 binary execname=/home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel,q=/RefAligner,p= Trying alternate binary: /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 Appending stdout to /root/MQR.stdout Appending stderr to /root/MQR.stdout

Oops! Something is wrong with your MQR running, please check!"

[yyx8671]

Hi @xiaoquexingchen,

Can you share your bnx file and the reference cmap file and then i can have a test.

[xiaoquexingchen]

Hi， It seems that your mailbox has set the size of the file to be transmitted. I switched to Baidu network disk sharing. Below is the link and extraction code. Thank you very much!

link: https://pan.baidu.com/s/1993SqPazmLR4NgdtOByn2w  extraction code: 5t2e 

------------------ 原始邮件 ------------------ 发件人: "AppliedBioinformatics/runBNG" @.>; 发送时间: 2021年4月11日(星期天) 下午4:02 @.>; @.**@.>; 主题: Re: [AppliedBioinformatics/runBNG] ERROR: sortBNX failed (#27)

Hi @xiaoquexingchen,

Can you share your bnx file and the reference cmap file and then i can have a test.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[yuxuanyuan]

seems i cannot open the page using the extraction code. Please check

[xiaoquexingchen]

Hi, I have resent the email to your mailbox yesterday, the link is: https://pan.baidu.com/s/1993SqPazmLR4NgdtOByn2w

The extraction code is: 5t2e

[yuxuanyuan]

Hi @xiaoquexingchen,

I tested your data using both Bionano Solve v3.5.1 and v3.6.1. It seems both of them work well on my server (I didn't wait for the completion and cancelled the run after a while),

The command line i used:

runBNG denovo -P saphyr -H yes -M yes -t ~/Tools/bionano/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/avx \
-s ~/Tools/bionano/tools/pipeline/Solve3.6.1_11162020 -b all.bnx -r hg19_DLE1_0kb_0labels.cmap -a 18 -z 3300 \
-T 8 -j 8 -o denovo_3.6.1/

The log information is:

  Prerun Tests:
    0 ERRORS
    0 WARNINGS

Hostname: xxxxx

Tools Version: N/A

Solve Version: N/A

Pipeline Version: 11646

RefAligner Version: 11643

Assembly ID: 535

  Pipeline start time: Mon Apr 12 11:11:37 2021

('checkScanScaling: autoNoise=', False)
Not performing autoNoise (see -y): setting doScanScale = False, hence not using _rescaled.bnx for BNX splits

Executing stage number 1 : AutoNoise + SplitBNX

Molecule Stats (/home/yyuan/runJobs/test/denovo_3.6.1/all.bnx):
Total number of molecules: 1147524
Total length (Mbp)       : 268845.841
Average length (kbp)     :    234.283
Molecule N50 (kbp)       :    233.550
Label density (/100kb)   :     17.395

Sorting /home/yyuan/runJobs/test/denovo_3.6.1/all.bnx into /home/yyuan/runJobs/test/denovo_3.6.1/all_sorted

 Starting Multi-Threaded Process: SortBNX
  Running 1 jobs with 8 threads, sleepTime=0.01
   START    1:                        SortBNX,   8 Thr,    1 R,    1 T,    0 F,    0 Q
   STOP     1:                        SortBNX,   8 Thr,    0 R,    1 T,    1 F,    0 Q  TotalTime= 0h 1.66m  RunTime= 0h 1.67m  CPUload=132% host=NA
 Finished Multi-Threaded Process: SortBNX

calculateNPairwise: Sorted and filtered file has 1147524 molecules, 47912204.0 sites, 268845841027.8 bp length total. blocks= 12 -> 12 (N=2)

Increasing number of blocks from 2 to 12

Splitting BNX

Splitting bnx file: /home/yyuan/runJobs/test/denovo_3.6.1/all_sorted.bnx

 Starting Multi-Threaded Process: SplitBNX
  Running 1 jobs with 8 threads, sleepTime=0.01
   START    1:                       SplitBNX,   8 Thr,    1 R,    1 T,    0 F,    0 Q
   STOP     1:                       SplitBNX,   8 Thr,    0 R,    1 T,    1 F,    0 Q  TotalTime= 0h 1.73m  RunTime= 0h 1.74m  CPUload=128% host=NA
 Finished Multi-Threaded Process: SplitBNX

Expecting:
/home/yyuan/runJobs/test/denovo_3.6.1/all_1_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_2_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_3_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_4_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_5_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_6_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_7_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_8_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_9_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_10_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_11_of_12.bnx, 
/home/yyuan/runJobs/test/denovo_3.6.1/all_12_of_12.bnx

Executing stage number 2 : Pairwise + Alignmolvref

 Starting Multi-Threaded Process: Pairwise
  Running 78 jobs with 8 threads, sleepTime=0.01
   START    1:               Pairwise 1 of 78,   8 Thr,    1 R,   78 T,    0 F,   77 Q

[xiaoquexingchen]

Hi, I tried again according to your order, but the error is still the same as before. Is it because my environment is not set up properly? But I can't find out that the piece is not set up properly. How should I check whether my environment is completely correct?

Enter "./runBNG denovo -P saphyr -H yes -M yes -t /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/avx -s /home/len/tools/pipeline/Solve3.6.1 _11162020 -b /home/len/all.bnx -r /home/len/hg19_DLE1_0kb_0labels.cmap -a 18 -z 3300 -T 8 -j 8 -o /root/denovo_3.6.1", it still reports an error:

Prerun Tests: 0 ERRORS 0 WARNINGS

Hostname: localhost.localdomain

Tools Version: N/A

Solve Version: N/A

Pipeline Version: 11646

RefAligner Version: 11643

Assembly ID: 478

Pipeline start time: Mon Apr 12 14:57:59 2021

('checkScanScaling: autoNoise=', False) Not performing autoNoise (see -y): setting doScanScale = False, hence not using _rescaled.bnx for BNX splits

Executing stage number 1: AutoNoise + SplitBNX

Molecule Stats (/root/denovo_3.6.1/all.bnx): Total number of molecules: 1147524 Total length (Mbp): 268845.841 Average length (kbp): 234.283 Molecule N50 (kbp): 233.550 Label density (/100kb): 17.395

Sorting /root/denovo_3.6.1/all.bnx into /root/denovo_3.6.1/all_sorted

Starting Multi-Threaded Process: SortBNX Running 1 jobs with 8 threads, sleepTime=0.01 START 1: SortBNX, 8 Thr, 1 R, 1 T, 0 F, 0 Q STOP 1: SortBNX, 8 Thr, 0 R, 1 T, 1 F, 0 Q TotalTime = 0h 0.54m RunTime = 0h 0.54m CPUload=76% host=NA Finished Multi-Threaded Process: SortBNX

ERROR: sortBNX failed. Check: /root/denovo_3.6.1/all.bnx

WARNING: missing xmap file: /root/denovo_3.6.1/contigs/exp_refineFinal1_sv/merged_smaps/exp_refineFinal1_merged.xmap

Pipeline end time: Mon Apr 12 14:58:35 2021 Elapsed time: 0.60m; 0.01h; 0.00d

Warning/Error messages: ('warning','Missing end marker in \"/root/denovo_3.6.1/all_sorted.stdout\" (found \"33472) failed\\n\" while expecting \"END of output \")\n') ('error','job has not completed, see stdout=\"/root/denovo_3.6.1/all_sorted.stdout\"') ('critical','ERROR: sortBNX failed. Check: /root/denovo_3.6.1/all.bnx')

Warning/Error summary: 1 warning(s) 1 critical(s) 1 error(s)

Pipeline has failed

[xiaoquexingchen]

And when I use all.bnx to test the MQR function, I also reported an error, but it can be run with the data in the examples of runBNG. All.bnx runs as follows: ./runBNG MQR -b /home/len/all.bnx -r /home/len/hg19_DLE1_0kb_0labels.cmap -z 3300 -R /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/avx/RefAligner -t 2 -m 2 -o /root/output -p MQR3

Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx/RefAligner compiled with SSE3=1, AVX=1, AVX2=0, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz __builtin_cpu_supports(avx) = 512 Appending stdout to /root/output/MQR3.stdout Appending stderr to /root/output/MQR3.stdout

Oops! Something is wrong with your MQR running, please check!

But I can run with the data in the examples. Why is this? Use the data in the examples to run as follows:

./runBNG MQR -b /root/runBNG-master/Examples/Molecules.bnx -r /root/runBNG-master/Examples/Test_BSPQI_20kb_5labels.cmap -R /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/ 1.0/RefAligner -t 2 -m 2 -o /root/runBNG-master/Examples -p singleMQR

Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 WARNING: host localhost.localdomain does not support AMD Zen2 or Intel Skylake avx2: switching to Haswell avx2 binary execname=/home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel,q=/RefAligner,p= Trying alternate binary: /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner Current binary /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner compiled with SSE3=1, AVX=1, AVX2=1, AVX512=0 /proc/cpuinfo: avx512 = 0, znver2 = 0: model name: Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz builtin_cpu_supports(avx2) = 1024, GNUC = 9 builtin_cpu_is(znver2) = 0 Appending stdout to /root/runBNG-master/Examples/singleMQR.stdout Appending stderr to /root/runBNG-master/Examples/singleMQR.stdout

False Positive Density (/100Kb) [FP]: 0.91 FP(%): 0.07 False Negative Rate (%/100) [FN]: 0.10 ScalingSD (Kb^1/2) [sd]: -0.07 SiteSD (Kb) [sf]: 0.17 RelativeSD [sr]: 0.03 Bpp: 497.19 N Molecules mapped to reference: 5000 Map rate (%): 75.12

[yyx8671]

Hi @xiaoquexingchen,

You may check your /root/denovo_3.6.1/all_sorted.stdout and find out what happened during the denovo assembly.

For the MQR, you may check /root/output/MQR3.stdout (maybe insufficent RAM was given. You may increase the RAM)

[xiaoquexingchen]

Hi, My /root/denovo_3.6.1/all_sorted.stdout file is shown as follows:

hostname=localhost.localdomain $ cd /root/runBNG-master; /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/avx/RefAligner -f -i /root/denovo_3.6.1/all.bnx -maxthreads 8 -merge -sort-idinc -bnx -o /root/denovo_3.6.1/all_sorted -minlen 120 -minsites 8 -maxmem 120 -maxvirtmem 0 -XmapStatWrite /root/denovo_3.6.1/molecule_stats.txt -stdout -stderr CompileDir= /home/users/sqa01/RefAlignerBuild/branches/11442.11643rel/11442.11643rel CompileCmd=/opt/gcc-8.4.0/bin/g++ -B/opt/binutils-2.30 -fopenmp -Ofast -mavx -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -DUSE_MFLOAT=1 -DUSE_EPOW=1 -I/opt/glibc-2.27/include -DUSE_STATIC -freciprocal-math -fno-signed-zeros -fno-trapping-math -fno-tree-vectorize -fno-lto-odr-type-merging -DRELEASE=1 -std=gnu++98 -Wl,--dynamic-linker=/opt/glibc-2.27/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/opt/glibc-2.31/lib -Wl,--rpath=/opt/gcc-8.4.0/lib64 -lrt -lmvec -L/opt/glibc-2.27/lib -L/opt/gcc-8.4.0/lib64 -static -static-libstdc++ -static-libgcc -s SVNversion=11643 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/11442/RefAligner.cpp 11543 2020-08-26 18:53:06Z tanantharaman $ FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 USE_MFLOAT=1 USE_EPOW=1 DEBUG=1 VERB=1 START TIME: Mon Apr 12 06:58:02 2021 OMP_WAIT_POLICY = PASSIVE euid= 0, TotalThreads=0,0 total Mem= 3.8389(available= 2.6211), VmRSS= 0.0089, free Swap= 1.9990, VmSwap= 0.0000, MaxMem= 120.0000, Margin= 0.0000 GB Changed -maxmem from 120.0000 Gb to 3.8389 Gb based on estimated system memory= 3.8389, Margin = 0.0000 GB WARNING: system has only 2 threads : reducing -maxthreads from 8 to 2 Reading input maps from /root/denovo_3.6.1/all.bnx Line 6 in /root/denovo_3.6.1/all.bnx: setting colors=1, colors_set=1, usecolor= 0 due to Label Channels: 1:

Label Channels: 1 input_bnx(/root/denovo_3.6.1/all.bnx):line 7:Changing global Nickase[0]=0xebe0b30(unknown) to 0xebe0b70(cttaag;green_01) Detected SNR information (QX11) on line 303 in /root/denovo_3.6.1/all.bnx Detected Intensity information (QX12) on line 304 in /root/denovo_3.6.1/all.bnx VmRSS= 3.5609, VmHWM= 3.6021 : CPU time= 18.571486, wall time= 21.796648 Finished parsing maps in /root/denovo_3.6.1/all.bnx: maps=1147524, sites=46764680, length= 268845841.028kb (avg= 234.283 kb, label density= 17.395 /100kb, N50= 0.2336 Mb): wall time= 32.158864 secs mapcorrect:malloc(1177433472) failed

hostname=localhost.localdomain $ cd /root/runBNG-master; /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/1.0/avx/RefAligner -f -i /root/denovo_3.6.1/all.bnx -maxthreads 2 -merge -sort-idinc -bnx -o /root/denovo_3.6.1/all_sorted -minlen 120 -minsites 8 -maxmem 120 -maxvirtmem 0 -XmapStatWrite /root/denovo_3.6.1/molecule_stats.txt -stdout -stderr CompileDir= /home/users/sqa01/RefAlignerBuild/branches/11442.11643rel/11442.11643rel CompileCmd=/opt/gcc-8.4.0/bin/g++ -B/opt/binutils-2.30 -fopenmp -Ofast -mavx -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -DUSE_MFLOAT=1 -DUSE_EPOW=1 -I/opt/glibc-2.27/include -DUSE_STATIC -freciprocal-math -fno-signed-zeros -fno-trapping-math -fno-tree-vectorize -fno-lto-odr-type-merging -DRELEASE=1 -std=gnu++98 -Wl,--dynamic-linker=/opt/glibc-2.27/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/opt/glibc-2.31/lib -Wl,--rpath=/opt/gcc-8.4.0/lib64 -lrt -lmvec -L/opt/glibc-2.27/lib -L/opt/gcc-8.4.0/lib64 -static -static-libstdc++ -static-libgcc -s SVNversion=11643 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/11442/RefAligner.cpp 11543 2020-08-26 18:53:06Z tanantharaman $ FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 USE_MFLOAT=1 USE_EPOW=1 DEBUG=1 VERB=1 START TIME: Mon Apr 12 07:07:57 2021 OMP_WAIT_POLICY = PASSIVE euid= 0, TotalThreads=0,0 total Mem= 3.8389(available= 3.0566), VmRSS= 0.0088, free Swap= 1.5293, VmSwap= 0.0000, MaxMem= 120.0000, Margin= 0.0000 GB Changed -maxmem from 120.0000 Gb to 3.8389 Gb based on estimated system memory= 3.8389, Margin = 0.0000 GB Reading input maps from /root/denovo_3.6.1/all.bnx Line 6 in /root/denovo_3.6.1/all.bnx: setting colors=1, colors_set=1, usecolor= 0 due to Label Channels: 1:

Label Channels: 1 input_bnx(/root/denovo_3.6.1/all.bnx):line 7:Changing global Nickase[0]=0xebbfb30(unknown) to 0xebbfb70(cttaag;green_01) Detected SNR information (QX11) on line 303 in /root/denovo_3.6.1/all.bnx Detected Intensity information (QX12) on line 304 in /root/denovo_3.6.1/all.bnx VmRSS= 3.4706, VmHWM= 3.4737 : CPU time= 18.236198, wall time= 22.209157 Finished parsing maps in /root/denovo_3.6.1/all.bnx: maps=1147524, sites=46764680, length= 268845841.028kb (avg= 234.283 kb, label density= 17.395 /100kb, N50= 0.2336 Mb): wall time= 30.547456 secs mapcorrect:malloc(1177433472) failed

My /root/output/MQR3.stdout file is shown as follows:

hostname=localhost.localdomain $ cd /root/runBNG-master; /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx/RefAligner -f -ref /home/len/hg19_DLE1_0kb_0labels.cmap -i /home/len/all.bnx -o /root/output/MQR3 -nosplit 2 -BestRef 1 -biaswt 0 -Mfast 0 -FP 1.5 -FN 0.15 -sf 0.2 -sd 0.0 -A 5 -outlier 1e-3 -outlierMax 40 -endoutlier 1e-4 -S -1000 -sr 0.03 -se 0.2 -MaxSF 0.25 -MaxSE 0.5 -resbias 4 64 -maxmem 2 -M 3 3 -minlen 150 -T 3.0303e-09 -maxthreads 2 -hashgen 5 3 2.4 1.5 0.05 5.0 1 1 3 -hash -hashdelta 10 -hashoffset 1 -hashmaxmem 2 -insertThreads 4 -maptype 0 -PVres 2 -PVendoutlier -AlignRes 2.0 -rres 0.9 -resEstimate -ScanScaling 2 -RepeatMask 5 0.01 -RepeatRec 0.7 0.6 1.4 -maxEnd 50 -usecolor 1 -stdout -stderr -randomize 1 -subset 1 5000 CompileDir= /home/users/sqa01/RefAlignerBuild/branches/11442.11643rel/11442.11643rel CompileCmd=/opt/gcc-8.4.0/bin/g++ -B/opt/binutils-2.30 -fopenmp -Ofast -mavx -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -DUSE_MFLOAT=1 -DUSE_EPOW=1 -I/opt/glibc-2.27/include -DUSE_STATIC -freciprocal-math -fno-signed-zeros -fno-trapping-math -fno-tree-vectorize -fno-lto-odr-type-merging -DRELEASE=1 -std=gnu++98 -Wl,--dynamic-linker=/opt/glibc-2.27/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/opt/glibc-2.31/lib -Wl,--rpath=/opt/gcc-8.4.0/lib64 -lrt -lmvec -L/opt/glibc-2.27/lib -L/opt/gcc-8.4.0/lib64 -static -static-libstdc++ -static-libgcc -s SVNversion=11643 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/11442/RefAligner.cpp 11543 2020-08-26 18:53:06Z tanantharaman $ FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 USE_MFLOAT=1 USE_EPOW=1 DEBUG=1 VERB=1 START TIME: Mon Apr 12 07:14:13 2021 OMP_WAIT_POLICY = (null) WARNING: -RepeatMask with -RepeatRec requires -extend 1 or -extend 2 (using -extend 1) euid= 0, TotalThreads=0,0 total Mem= 3.8389(available= 3.1631), VmRSS= 0.0089, free Swap= 1.5156, VmSwap= 0.0000, MaxMem= 2.0000, MaxVirtMem= -1.0000, Margin= 0.0000 GB MemSize = 4121952256, SwapSize = 1627389952 : Setting MaxVirtMem = 3.5156 Gb (MaxMem= 2.0000 Gb) hash_prefix = /root/output/MQR3_p3942_id0 hash_filename = /root/output/MQR3_p3942_id0.hashbin Reading input maps from /home/len/all.bnx Line 6 in /home/len/all.bnx: setting colors=1, colors_set=1, usecolor= 0 due to Label Channels: 1:

Label Channels: 1 input_bnx(/home/len/all.bnx):line 7:Changing global Nickase[0]=0xdf80fc0(unknown) to 0xdf81000(cttaag;green_01) Detected SNR information (QX11) on line 303 in /home/len/all.bnx Detected Intensity information (QX12) on line 304 in /home/len/all.bnx VmRSS= 3.3296, VmHWM= 3.4654 : CPU time= 21.045911, wall time= 17.243901 Finished parsing maps in /home/len/all.bnx: maps=1147524, sites=46764680, length= 268845841.028kb (avg= 234.283 kb, label density= 17.395 /100kb, N50= 0.2336 Mb): wall time= 30.449793 secs mapcorrect:malloc(1177433472) failed

hostname=localhost.localdomain $ cd /root/runBNG-master; /home/len/tools/pipeline/Solve3.6.1_11162020/RefAligner/11442.11643rel/avx2/RefAligner -f -ref /home/len/hg19_DLE1_0kb_0labels.cmap -i /home/len/all.bnx -o /root/output/MQR3 -nosplit 2 -BestRef 1 -biaswt 0 -Mfast 0 -FP 1.5 -FN 0.15 -sf 0.2 -sd 0.0 -A 5 -outlier 1e-3 -outlierMax 40 -endoutlier 1e-4 -S -1000 -sr 0.03 -se 0.2 -MaxSF 0.25 -MaxSE 0.5 -resbias 4 64 -maxmem 2 -M 3 3 -minlen 150 -T 3.0303e-09 -maxthreads 2 -hashgen 5 3 2.4 1.5 0.05 5.0 1 1 3 -hash -hashdelta 10 -hashoffset 1 -hashmaxmem 2 -insertThreads 4 -maptype 0 -PVres 2 -PVendoutlier -AlignRes 2.0 -rres 0.9 -resEstimate -ScanScaling 2 -RepeatMask 5 0.01 -RepeatRec 0.7 0.6 1.4 -maxEnd 50 -usecolor 1 -stdout -stderr -randomize 1 -subset 1 5000 CompileDir= /home/users/sqa01/RefAlignerBuild/branches/11442.11643rel/11442.11643rel CompileCmd=/opt/gcc-9.3.0/bin/g++ -B/opt/binutils-2.34 -fopenmp -Ofast -march=haswell -mavx2 -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -DUSE_MFLOAT=1 -DUSE_EPOW=1 -I/opt/glibc-2.31/include -DUSE_STATIC -freciprocal-math -fno-signed-zeros -fno-trapping-math -fno-tree-vectorize -mtune=haswell -fno-lto-odr-type-merging -DRELEASE=1 -std=gnu++98 -Wl,--dynamic-linker=/opt/glibc-2.31/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/opt/glibc-2.31/lib -Wl,--rpath=/opt/gcc-9.3.0/lib64 -lrt -lmvec -L/opt/glibc-2.31/lib -L/opt/gcc-9.3.0/lib64 -static -static-libstdc++ -static-libgcc -s SVNversion=11643 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/11442/RefAligner.cpp 11543 2020-08-26 18:53:06Z tanantharaman $ FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 USE_MFLOAT=1 USE_EPOW=1 DEBUG=1 VERB=1 START TIME: Mon Apr 12 07:24:32 2021 OMP_WAIT_POLICY = (null) WARNING: -RepeatMask with -RepeatRec requires -extend 1 or -extend 2 (using -extend 1) euid= 0, TotalThreads=0,0 total Mem= 3.8389(available= 3.1270), VmRSS= 0.0090, free Swap= 1.5605, VmSwap= 0.0000, MaxMem= 2.0000, MaxVirtMem= -1.0000, Margin= 0.0000 GB MemSize = 4121952256, SwapSize = 1675624448 : Setting MaxVirtMem = 3.5605 Gb (MaxMem= 2.0000 Gb) hash_prefix = /root/output/MQR3_p4802_id0 hash_filename = /root/output/MQR3_p4802_id0.hashbin Reading input maps from /home/len/all.bnx Line 6 in /home/len/all.bnx: setting colors=1, colors_set=1, usecolor= 0 due to Label Channels: 1:

Label Channels: 1 input_bnx(/home/len/all.bnx):line 7:Changing global Nickase[0]=0xde4f0d0(unknown) to 0xde4f110(cttaag;green_01) Detected SNR information (QX11) on line 303 in /home/len/all.bnx Detected Intensity information (QX12) on line 304 in /home/len/all.bnx VmRSS= 3.5660, VmHWM= 3.5744 : CPU time= 20.674545, wall time= 15.712086 Finished parsing maps in /home/len/all.bnx: maps=1147524, sites=46764680, length= 268845841.028kb (avg= 234.283 kb, label density= 17.395 /100kb, N50= 0.2336 Mb): wall time= 27.907323 secs mapcorrect:malloc(1177433472) failed

I can't understand the problem too well, do I need more running memory? I did set the running memory a bit small at the time. Thanks

[yyx8671]

Hi @xiaoquexingchen,

It seems your server has limited RAM and CPUs and your jobs were crashed due to insufficient RAM. You may check with your admin and see if you can run heavy workload jobs on your server. All the best!

[xiaoquexingchen]

Hi, Thank you very much for your timely reply every time, I will try to change the running memory.

Thanks