Interthread memory fail

[GoogleCodeExporter]

As long as the problem exists with interthreaded memory, ScaleMM can not be 
used in a productive environment. That is a big problem for me, because from 
the viewpoint of speed ScaleMM is really convincing. 

I made a test based on the simple thread demo of scaleMM3 with additional 
creating some interfacedobject (with reference counter). When sharing the 
interface pointer to another thread and free the object in context of this 
thread then ScaleMM fail with an exception (tested with ScaleMM2 and ScaleMM3). 
The same programm executing with the Delphi memory manager or with FastMM4 runs 
without problems.

Please clean up the code and made it threadsafe in case of interthreaded memory 
usage.

If you need more informations or an example then give me an feedback.

Regards
Maxx (DE)

Original issue reported on code.google.com by mxxa2a...@gmail.com on 26 Feb 2012 at 9:08

[GoogleCodeExporter]

Yes, please add a demo program, this will speedup the bugfixing a lot.

But I can't promise anything however, I tried to finish SMM2 but still contains 
a hard to find bug (and my spare time is low so very frustrating and 
demotivating).

Don't know if SMM3 will be continued because of disappointing results. 

Interthread memory is very difficult because of thread safety and speed/scaling 
issues (I don't want to lock everytime, like FastMM does)

Original comment by andre.mussche on 27 Feb 2012 at 6:45

[GoogleCodeExporter]

Hi Andre,
thanks for your reply.

I will post a demo, but actually I have also not so much time. Please have a 
couple of days patience.

But keep in mind: To made a program threadsafe it's not a question of 
debugging, but a question of codestyle and advisement.

As you know you need to place a CriticalSections (or 
MultipleReadExclusiveWriteSynchronizer or similar) around each memory 
allocation table. ALL access to the tables need to enter the critical section 
(identified by ThreadID). If a thread will free a memory how's not a part of 
the own table (detected by ThreadID I think) then the thread needs enter the 
CrticalSection of the source thread memory table and free the memory here. 
Until the source thread memory table is blocked by the critical section this 
thead can't access to the own table (this is a speed penalty).

I'm looking forward that you will solve the problem.

Original comment by mxxa2a...@gmail.com on 27 Feb 2012 at 7:54

[GoogleCodeExporter]

Yes I know, and I don't do "trial and error" programming...
It should be threadsafe already, but somehow it has some weird and hard to find 
bugs.
Adding more locking will make it not scale anymore (or at least less). 
I will concentrate my time (if any) on SMM2 first.

Original comment by andre.mussche on 27 Feb 2012 at 9:42

[GoogleCodeExporter]

Here is the simple Demo with interthreaded memory allocations.
I hope it helps a little bit ...

Original comment by mxxa2a...@gmail.com on 28 Feb 2012 at 8:18

Attachments:

• Demo01.zip

[GoogleCodeExporter]

Thanks, will look at it tomorrow
Maybe I can release a stable version of SMM2 soon (already fixed some small 
stuff)

Original comment by andre.mussche on 28 Feb 2012 at 8:33

[GoogleCodeExporter]

I try to take a look at the code, but the public release ScaleMM_v205.zip is 
not equal with the branche http://scalemm.googlecode.com/svn/branches/version2 
(rev 81).
I find out, that the release ScaleMM_v205 make an exception with my Demo01, but 
the branches\..\version2 not (but I tested on a single-core PC).
What are the latest sources of ScaleMM2??

Original comment by mxxa2a...@gmail.com on 28 Feb 2012 at 8:52

[GoogleCodeExporter]

Latest version (local) seems to work OK.
When I fix the latest (?) bug I will release a new version (and update the 
source in the v2 branch)

Latest version in v2 branch can contain some small fixes, but not stable yet so 
I did not update a new version.zip (because I hoped I could make it stable in a 
short period of time... which failed. a second try now...)

Original comment by andre.mussche on 29 Feb 2012 at 7:38

[GoogleCodeExporter]

see the version2 branch for a new version, which should be (more) stable

Original comment by andre.mussche on 7 Mar 2012 at 7:48

• Changed state: Fixed

[GoogleCodeExporter]

I test the version2 again. Now it is much more stable but not in all 
situations. 
If I use V2 in a huge application (EXE+4DLL ~700.000 code lines) than is it 
stable for some time. After one hour (or so) I get an exception inside a 
FreeMem function. With other memorymanagers (FastMM) everything is fine. I 
thing you need another try ...

Original comment by mxxa2a...@gmail.com on 20 Mar 2012 at 8:35

[GoogleCodeExporter]

Ooops: Just today you made some more changes in V2. So I need to make my test 
again. I give a feedback after that... 

Original comment by mxxa2a...@gmail.com on 20 Mar 2012 at 8:40

[GoogleCodeExporter]

Here is my feedback for scalemm V2 of 20.03.12: At first look everything is 
fine. But after 15 minutes I get an exception in ScaleMM_FreeMem (in context of 
mainthread). After that the application goes ahead (no blocker exception).

Another thing are the exceptions when closing the application. A lot of 
exceptions are at the call FreeLibraray of dynamically linked DLL. It seems to 
be a problem with the memory alloced by a thread inside the Dll. 

Question: Is ScaleMM shared with the DLL? I think it is basically needed 
because to map strings between EXE and DLL...  

Original comment by mxxa2a...@gmail.com on 21 Mar 2012 at 7:53

[GoogleCodeExporter]

You need some kind of "ShareMem" when you want to share data between dll and 
exe.
Don't know if I can make a ScaleShareMem, but will take a look at it.
Can you make a small test project to test your AV?

Original comment by andre.mussche on 21 Mar 2012 at 8:00