gnudatalanguage / gdl

GDL - GNU Data Language
GNU General Public License v2.0
274 stars 61 forks source link

shmmap causes linux to reboot if gdl was not started with cpu,tpool_nthreads=1 #1834

Open brandy125 opened 4 months ago

brandy125 commented 4 months ago

I am testing the version 1.0.5 with shmmap and heavy load.

Sharing variables with approx. 5GB in size to 16 CPU cores using shmmap. Everything works very nice on Suse Tumbleweed inside Parallels on a Mac and on a normal PC with Intel i9-14000K as long as I put in my startup script the following two lines:

cpu,TPOOL_MIN_ELTS=2e8 cpu,tpool_nthreads=1

If I remove these lines the PC reboots after some minutes (on the MAC I did not test this configuration).

I tested the program for hours with cpu,tpool_nthreads=1 and all worked fine!

GDL> !gdl
{
    "RELEASE": "1.0.5",
    "BUILD_DATE": "May 21 2024",
    "EPOCH": 1716260400,
    "GDL_USE_DSFMT": 1,
    "GDL_USE_WX": 1,
    "GDL_POSIX": 1
}

Screenshot_2024-05-21 17 48 38_DcP6L3 Screenshot_2024-05-21 17 48 15_YDJ40N Screenshot_2024-05-21 17 47 57_lLgn15

GillesDuvert commented 4 months ago

@brandy125 what is the behaviour of gdl --smart-tpool ?

GillesDuvert commented 2 months ago

Coming back to this --- the simple code for shmap, etc in GDL is certainly not thread-proof. I would have thought that it could not be a problem for readonly shared memory (the usual case). Still, is it a shared memory problem or an accumulation of threads waiting for access to the shared memory? gdl --smart-tpool may be of help.