search

GeneAssembly / biosal

biosal is a distributed BIOlogical Sequence Actor Library. THIS IS A MIRROR.
BSD 2-Clause "Simplified" License
6 stars 1 forks source link

performance is better with symmetric actors #638

Closed sebhtml closed 10 years ago

sebhtml commented 10 years ago

see #627 test-spate-627-2048x16-12 18 minutes, 17.632812 seconds (pretty fast) test-spate-627-2048x16-17 21 minutes, 43.527222 seconds

define BSAL_MAXIMUM_GRAPH_STORE_COUNT ((16 - 1) * 1024)

became

define BSAL_MAXIMUM_GRAPH_STORE_COUNT ((16 - 1) * 512)

presumably, this should be dynamic:

2048 -> 1024_15 1024 -> 512_15 512 -> 256*15

and so on

on bgq only.

on other systems, just use symmetric actors anyway

sebhtml commented 10 years ago

@hubot says: implemented in commit https://github.com/sebhtml/biosal/commit/fb9a0d6cc70a8df0241f3bcb702f0aa37fa2c0d6 by @sebhtml !

sebhtml commented 10 years ago

JobName Machine cetus the great

AllocationStatus Path Commit [boisvert@cetuslac1 biosal]$ git log | head -n1 commit 27bb8427bbc1310db3222a8c60311b5691222ab7 build with CONFIG_PAMI=n

Toolchain /soft/compilers/ibmcmp-may2014/vac/bg/12.1/bin/.orig/bgxlc_r:

Script [boisvert@cetuslac1 biosal-tests]$ cat test-spate-638-2048x16-2.sh

!/bin/bash

qsub \ --env PAMID_THREAD_MULTIPLE=1 \ -A CompBIO \ -n 2048 \ -t 00:30:00 \ -O test-spate-638-2048x16-2 \ --mode c1 \ spate -print-load -threads-per-node 16 \ -k 43 Iowa_Continuous_Corn/*.fastq \ -o test-spate-638-2048x16-2

Submission [boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-2.sh 323793 (canceled)

http://status.alcf.anl.gov/cetus/activity

MachineUtilization ComputationLoad RunningTime MemoryUtilization Checksum GoodComments BadComments NeutralComments

sebhtml commented 10 years ago

[boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-3.sh 324003 (out of mem)

[boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-4.sh 324007 (out of memory)

sebhtml commented 10 years ago

JobName Machine cetus for testing

AllocationStatus Path [boisvert@cetuslac1 biosal]$ pwd /projects/CompBIO/Projects/biosal-tests/biosal

Commit [boisvert@cetuslac1 biosal]$ git log | head -n1 commit 7defd15bb4c8a9b83c4988c53e98305a448f8fe4

Toolchain Script [boisvert@cetuslac1 biosal-tests]$ cat test-spate-638-2048x16-4.sh

!/bin/bash

qsub \ --env PAMID_THREAD_MULTIPLE=1 \ -A CompBIO \ -n 2048 \ -t 00:30:00 \ -O test-spate-638-2048x16-4 \ --mode c1 \ spate -print-load -threads-per-node 16 \ -k 43 Iowa_Continuous_Corn/*.fastq \ -o test-spate-638-2048x16-4

Submission [boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-4.sh 324007

MachineUtilization ComputationLoad RunningTime MemoryUtilization Checksum GoodComments BadComments out of memory

NeutralComments

sebhtml commented 10 years ago

[boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-5.sh 324018 PASSED

sebhtml commented 10 years ago

JobName 2048x16 cetus

Machine cetus

AllocationStatus Path [boisvert@cetuslac1 biosal-tests]$ pwd /projects/CompBIO/Projects/biosal-tests

Commit [boisvert@cetuslac1 biosal]$ git log | head -n1 commit f1db383279e32c40afe37522a9a09ef4149b6ea7

Toolchain /soft/compilers/ibmcmp-may2014/vac/bg/12.1/bin/.orig/bgxlc_r:

Script [boisvert@cetuslac1 biosal-tests]$ cat test-spate-638-2048x16-5.sh

!/bin/bash

qsub \ --env PAMID_THREAD_MULTIPLE=1 \ -A CompBIO \ -n 2048 \ -t 00:30:00 \ -O test-spate-638-2048x16-5 \ --mode c1 \ spate -print-load -threads-per-node 16 \ -k 43 Iowa_Continuous_Corn/*.fastq \ -o test-spate-638-2048x16-5

Submission [boisvert@cetuslac1 biosal-tests]$ ./test-spate-638-2048x16-5.sh 324018

MachineUtilization ComputationLoad [boisvert@cetuslac1 biosal-tests]$ grep LOAD test-spate-638-2048x16-5.output |grep COMP|grep -v " s "|tail [thorium] node/1525 COMPUTATION LOAD 0.39 [thorium] node/1799 COMPUTATION LOAD 0.38 [thorium] node/1563 COMPUTATION LOAD 0.40 [thorium] node/1272 COMPUTATION LOAD 0.40 [thorium] node/1399 COMPUTATION LOAD 0.40 [thorium] node/1072 COMPUTATION LOAD 0.41 [thorium] node/1950 COMPUTATION LOAD 0.41 [thorium] node/1375 COMPUTATION LOAD 0.41 [thorium] node/1258 COMPUTATION LOAD 0.38 [thorium] node/1407 COMPUTATION LOAD 0.42

RunningTime [boisvert@cetuslac1 biosal-tests]$ grep TIMER test-spate-638-2048x16-5.output TIMER [Load input / Count input data] 43.176437 seconds TIMER [Load input / Distribute input data] 2 minutes, 48.987137 seconds TIMER [Load input] 3 minutes, 32.163589 seconds TIMER [Build assembly graph / Distribute vertices] 6 minutes, 54.535645 seconds TIMER [Build assembly graph / Distribute arcs] 8 minutes, 11.946838 seconds TIMER [Build assembly graph] 15 minutes, 6.482483 seconds TIMER [Run actor computation] 18 minutes, 45.686035 seconds

MemoryUtilization [boisvert@cetuslac1 biosal-tests]$ grep ByteCount test-spate-638-2048x16-5.output|awk '{print $9}'|sort -r -n|head -n1 16433311744

Checksum [boisvert@cetuslac1 biosal-tests]$ sha1sum test-spate-638-2048x16-5/coverage_distribution.txt-canonical 01a293db48518190038eaddbaed8a47ca0323fc7 test-spate-638-2048x16-5/coverage_distribution.txt-canonical

GoodComments BadComments NeutralComments

sebhtml commented 10 years ago

size:

GRAPH bsal_assembly_graph_builder/1001469 -> 141189180698 vertices, 253689215456 vertex observations, and 140183562558 arcs. (from #627)