moebius_3 test case

[jodavies]

I often have fails on this test with tform due to its memory consumption with tform -w8, which is 18GB or so. I see it is already disabled on github due to fails, presumably for the same reason. Maybe this one is more reliable if it only runs in serial tests?

Thanks, Josh.

[tueda]

Maybe a better solution would be to run the test only if the total memory size of the host is large enough (check.rb should provide a new function, say, total_memory).

For reference, here is a memory profiling result (v5.0.0-beta.1-20-gffcea7d):

Peak memory information by Massif

``` valgrind --tool=massif ./build/sources/vorm -D TEST=moebius_3 check/features.frm ``` ``` -------------------------------------------------------------------------------- n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B) -------------------------------------------------------------------------------- 46 12,893,571,748 5,675,700,208 5,675,682,798 17,410 0 100.00% (5,675,682,798B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc. ->100.00% (5,675,608,864B) 0x23224C: Malloc1 (tools.c:2322) | ->37.89% (2,150,725,584B) 0x20C49D: AllocSort (setfile.c:916) | | ->37.49% (2,127,868,504B) 0x20CD49: AllocSetups (setfile.c:575) | | | ->37.49% (2,127,868,504B) 0x20D62D: MakeSetupAllocs (setfile.c:1090) | | | ->37.49% (2,127,868,504B) 0x21AD10: main (startup.c:1666) | | | | | ->00.40% (22,857,080B) in 1+ places, all below ms_print's threshold (01.00%) | | | ->37.84% (2,147,516,440B) 0x1FED41: Moebius (reken.c:3745) | | ->37.84% (2,147,516,440B) 0x18F6C0: Normalize (normal.c:1976) | | ->37.84% (2,147,516,440B) 0x1E55F1: Generator (proces.c:3272) | | ->37.84% (2,147,516,440B) 0x1E6DA0: Generator (proces.c:4048) | | ->37.84% (2,147,516,440B) 0x1E7205: Generator (proces.c:4216) | | ->37.84% (2,147,516,440B) 0x1E876F: Processor (proces.c:406) | | ->37.84% (2,147,516,440B) 0x15268B: DoExecute (execute.c:858) | | ->37.84% (2,147,516,440B) 0x182387: ExecModule (module.c:289) | | ->37.84% (2,147,516,440B) 0x1DD5DE: PreProcessor (pre.c:1041) | | ->37.84% (2,147,516,440B) 0x21ADA6: main (startup.c:1688) | | | ->18.52% (1,051,335,072B) 0x20CB4F: AllocSetups (setfile.c:531) | | ->18.52% (1,051,335,072B) 0x20D62D: MakeSetupAllocs (setfile.c:1090) | | ->18.52% (1,051,335,072B) 0x21AD10: main (startup.c:1666) | | | ->05.64% (320,000,000B) 0x20C8B0: AllocSetups (setfile.c:447) | | ->05.64% (320,000,000B) 0x20D62D: MakeSetupAllocs (setfile.c:1090) | | ->05.64% (320,000,000B) 0x21AD10: main (startup.c:1666) | | | ->00.11% (6,031,768B) in 1+ places, all below ms_print's threshold (01.00%) | ->00.00% (73,934B) in 1+ places, all below ms_print's threshold (01.00%) ``` ``` valgrind --tool=massif ./build/sources/tvorm -w8 -D TEST=moebius_3 check/features.frm ``` ``` -------------------------------------------------------------------------------- n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B) -------------------------------------------------------------------------------- 69 51,570,004,826 19,216,824,832 19,216,659,806 165,026 0 100.00% (19,216,659,806B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc. ->100.00% (19,216,581,616B) 0x240BA3: Malloc1 (tools.c:2322) | ->44.70% (8,590,000,168B) 0x20ACB2: Moebius (reken.c:3745) | | ->44.70% (8,590,000,168B) 0x195A09: Normalize (normal.c:1976) | | ->44.70% (8,590,000,168B) 0x1EEA9D: Generator (proces.c:3272) | | ->44.70% (8,590,000,168B) 0x1F0846: Generator (proces.c:4048) | | ->44.70% (8,590,000,168B) 0x1F0E0F: Generator (proces.c:4216) | | ->44.70% (8,590,000,168B) 0x253F20: RunThread (threads.c:1393) | | ->44.70% (8,590,000,168B) 0x4E13AC2: start_thread (pthread_create.c:442) | | ->44.70% (8,590,000,168B) 0x4EA4A03: clone (clone.S:100) | | | ->27.01% (5,191,035,112B) 0x218A0D: AllocSort (setfile.c:916) | | ->14.57% (2,799,124,352B) 0x252E4F: InitializeOneThread (threads.c:733) | | | ->14.57% (2,799,124,352B) 0x253BBE: RunThread (threads.c:1268) | | | ->14.57% (2,799,124,352B) 0x4E13AC2: start_thread (pthread_create.c:442) | | | ->14.57% (2,799,124,352B) 0x4EA4A03: clone (clone.S:100) | | | | | ->11.73% (2,254,767,800B) 0x2191A0: AllocSetups (setfile.c:575) | | | ->11.73% (2,254,767,800B) 0x219A3C: MakeSetupAllocs (setfile.c:1090) | | | ->11.73% (2,254,767,800B) 0x2286A9: main (startup.c:1666) | | | | | ->00.71% (137,142,960B) in 1+ places, all below ms_print's threshold (01.00%) | | | ->14.99% (2,880,000,000B) 0x2527A1: InitializeOneThread (threads.c:586) | | ->13.32% (2,560,000,000B) 0x253BBE: RunThread (threads.c:1268) | | | ->13.32% (2,560,000,000B) 0x4E13AC2: start_thread (pthread_create.c:442) | | | ->13.32% (2,560,000,000B) 0x4EA4A03: clone (clone.S:100) | | | | | ->01.67% (320,000,000B) 0x258BB6: StartAllThreads (threads.c:299) | | ->01.67% (320,000,000B) 0x22870B: main (startup.c:1675) | | | ->08.53% (1,638,445,056B) 0x252486: InitializeOneThread (threads.c:529) | | ->08.21% (1,577,005,056B) 0x258BB6: StartAllThreads (threads.c:299) | | | ->08.21% (1,577,005,056B) 0x22870B: main (startup.c:1675) | | | | | ->00.32% (61,440,000B) in 1+ places, all below ms_print's threshold (01.00%) | | | ->03.34% (641,344,128B) 0x2530AD: MakeThreadBuckets (threads.c:921) | | ->03.34% (641,344,128B) 0x258CE1: StartAllThreads (threads.c:309) | | ->03.34% (641,344,128B) 0x22870B: main (startup.c:1675) | | | ->01.26% (241,920,000B) 0x252110: InitializeOneThread (threads.c:429) | | ->01.26% (241,920,000B) 0x25710D: RunSortBot (threads.c:1926) | | ->01.26% (241,920,000B) 0x4E13AC2: start_thread (pthread_create.c:442) | | ->01.26% (241,920,000B) 0x4EA4A03: clone (clone.S:100) | | | ->00.18% (33,837,152B) in 1+ places, all below ms_print's threshold (01.00%) | ->00.00% (78,190B) in 1+ places, all below ms_print's threshold (01.00%) ```

[jodavies]

In addition, currently when I run make check after control returns to my shell, there are some leftover processes which are stuck until I kill them. Maybe this is messing up the CI also.

[tueda]

I think you have a test case that gives TIMEOUT (Issue7_3). It seems that check.rb fails to kill timed-out processes. I was aware of it with WSL2 but thought it might be a WSL-specific problem.

Probably, you are working on a real Linux and hit the same problem. So, I have created an issue for it: #476