search

andremussche / scalemm

Fast scaling memory manager for Delphi
https://code.google.com/p/scalemm/
Other
98 stars 22 forks source link

memory is growing until the thread is close :( #24

Closed Zeus64 closed 8 years ago

Zeus64 commented 8 years ago

i create an application with 10 threads, each thread doing in loop some database queries (and the memory allocation need to handle the response), and after 1 or 2 min i have out of memory :(

if i stop the loop and close all thread before the out of memory error then the memory go down to it's starting level (around 12mb)

andremussche commented 8 years ago

can you also reproduce it with a small test application? (with no database or maybe with sqllite?) what do you do with the result of the queries: it is processed/used in an other thread?

2015-11-06 15:15 GMT+01:00 loki5100 notifications@github.com:

i create an application with 10 threads, each thread doing in loop some database queries (and the memory allocation need to handle the response), and after 1 or 2 min i have out of memory :(

if i stop the loop and close all thread before the out of memory error then the memory go down to it's starting level (around 12mb)

— Reply to this email directly or view it on GitHub https://github.com/andremussche/scalemm/issues/24.

Zeus64 commented 8 years ago

you can see the application here (svn): https://svn.code.sf.net/p/alcinoe/code/

in the drectory /demo/ALDatabaseBenchmark

it's also work with sqllite3, memcached, mongodb, firebird, mysql, etc.. doesn't really matter, simply launch with loop and select 10 or 20 for "Nb of thread". it's more easy to see the bug if the query return lot of row (around 10 for exemple will be enalf)

you will need devExpress to compile the demo.

it's was just an application to with i switch fastmm to scalemm, so it's was not really build to test scalemm, but it's show that moving from fastmm to scalemm result with bug :(

Zeus64 commented 8 years ago

i m soo sorry for the false issue, it's was not connected to scalemm at all :( but it's was connected to my code instead :( sorry !

anyway the good new is that now the application is 5 time (yes 5 times) more faster with scalemm than without (on multithread)

i didn't know, but with fastmm, more processor cores you will have, more slow will be your multithread application !! with fastmm, by setting the processor afinity to only one processor (in the task manager), i succed to have performance close to scalemm on 8 cores. fastmm with 8 cores, it's 5 times more slower than scalemm on 8 cores

andremussche commented 8 years ago

thanks for the response anyway :)

About the speed of fastmm on a single processor: can you post some times? Because it is interresting on how fastmm performs unpinned and pinned on a processor for other users and scenarios So please the times of fastmm unpinned, fastmm pinned, and scalemm?

Zeus64 commented 8 years ago

Ok, so i didn't use a dedicated application, simply use databaseBenchmark

https://svn.code.sf.net/p/alcinoe/code/ in the drectory /demo/ALDatabaseBenchmark

who work on multithread to stress a database server.

0.12ms by query ScaleMM2

0.15ms by query ScaleMM2

0.19ms by query ScaleMM2

0.15ms by query ScaleMM2

andremussche commented 8 years ago

thanks for the screenshots! Did you run the database too on the same computer? Because I don't understand the high cpu when running on one core...

CPU affinity to one (!) core makes it single threaded, right? so that explains the same speed (0.15ms) for both FastMM en SMM when running on one core. (so ScaleMM is fast in single threaded mode too, nice! :) )

2015-11-23 16:40 GMT+01:00 loki5100 notifications@github.com:

Ok, so i didn't use a dedicated application, simply use databaseBenchmark

https://svn.code.sf.net/p/alcinoe/code/ in the drectory /demo/ALDatabaseBenchmark

who work on multithread to stress a database server.

0.12ms by query [image: ScaleMM2] https://camo.githubusercontent.com/c55387315e18f843cd933329162bcfb376ebea38/687474703a2f2f63312e737461746963666c69636b722e636f6d2f312f3737352f32323837373731393935395f616363353037363033635f622e6a7067

0.15ms by query [image: ScaleMM2] https://camo.githubusercontent.com/0d895f9c1827309e802b2319b7f64c40358c58fc/687474703a2f2f63322e737461746963666c69636b722e636f6d2f362f353830372f32323631383536373735335f376363333138326261375f622e6a7067

0.19ms by query [image: ScaleMM2] https://camo.githubusercontent.com/ab876d710a1e86fca1daa8f246ccb8d6a0d38e4d/687474703a2f2f63312e737461746963666c69636b722e636f6d2f312f3735382f32333234353639303232355f333265653337393866645f622e6a7067

0.15ms by query [image: ScaleMM2] https://camo.githubusercontent.com/0efb9920432354adae821d291eaef8d4ac94db08/687474703a2f2f63322e737461746963666c69636b722e636f6d2f362f353637372f32323934393833393235305f353137326331336331645f622e6a7067

— Reply to this email directly or view it on GitHub https://github.com/andremussche/scalemm/issues/24#issuecomment-158972156 .