Tokens are allocated using malloc but not registered as roots to the GC

[GoogleCodeExporter]

Since Tokens seems to have pointers to the GC heap and they have a custom
allocator that gets the memory from the C heap using malloc(), they should
be registered as roots to the GC to prevent premature freeing of data which
only reference are from Token instances (if there is any).

I guess it could be better to just use the GC heap for tokens, unless there
is a real (and necessary) performance gain, to avoid hard to figure-out
memory errors.

Original issue reported on code.google.com by llu...@gmail.com on 6 Sep 2009 at 9:00

[GoogleCodeExporter]

Original comment by aziz.koe...@gmail.com on 23 Feb 2010 at 6:58

• Changed state: Accepted

[GoogleCodeExporter]

Issue 12 has been merged into this issue.

Original comment by aziz.koe...@gmail.com on 23 Feb 2010 at 10:43

[GoogleCodeExporter]

Fixed in:
http://github.com/azizk/dil/commit/9fa0bbee1d2e736898e74b788ea9a7136ed74f39

I disabled the custom allocators with version statements. Could you test this 
again?

Original comment by aziz.koe...@gmail.com on 23 Feb 2010 at 11:00

• Changed state: Fixed

[GoogleCodeExporter]

I'm reopening this issue. I noticed that there is indeed a huge performance 
gain, if 
you allocate the tokens with malloc.

Could you perhaps help me out here? I'd like to know how to fix this the 
correct way.

Original comment by aziz.koe...@gmail.com on 26 Feb 2010 at 7:09

• Changed state: Started

[GoogleCodeExporter]

Sure, I think you should use tango.core.gc.GC.addRange(p, p + size) to register 
the
malloc()ed memory chunk to the GC, so it can scan it. That should be enough.

Original comment by llu...@gmail.com on 26 Feb 2010 at 9:04

[GoogleCodeExporter]

Thanks, but I have trouble getting this to work.

I import GC from `import tango.core.Memory`, and insert this line in 
`Token.new(size_t 
size)`: `GC.addRange(p, size);` or `GC.addRange(p, cast(size_t)(p+size));`

When I run dil, I get a segmentation fault at the end of the program. Here is 
the backtrace 
from gdb:
{{{
Program received signal SIGSEGV, Segmentation fault.
0x080c6339 in _D2rt2gc5basic3gcx3Gcx4markMFPvPvZv ()

#0  0x080c6339 in _D2rt2gc5basic3gcx3Gcx4markMFPvPvZv ()
#1  0x080c665c in _D2rt2gc5basic3gcx3Gcx11fullcollectMFPvZk ()
#2  0x080c649a in _D2rt2gc5basic3gcx3Gcx16fullcollectshellMFZk ()
#3  0x080c5384 in _D2rt2gc5basic3gcx2GC11fullCollectMFZv ()
#4  0x080c3eeb in gc_term ()
#5  0x080b0c53 in _D2rt8compiler3dmd2rt6dmain24mainUiPPaZi6runAllMFZv ()
#6  0x080b0b69 in _D2rt8compiler3dmd2rt6dmain24mainUiPPaZi7tryExecMFDFZvZv ()
#7  0x080b0b1a in main ()
}}}

The problem doesn't get solved by inserting the line `GC.removeRange(p);` in 
`Token.delete
(void* p)`.

Another issue is that dil doesn't seem to work with custom allocation at all. 
Parsing some 
source files throw the following exception: tango.core.Exception.IOException: 
<console> :: 
Bad address

I have no idea why. This didn't happen with Tango 0.99.8. So the only way to 
make dil work 
reliably now, is to let the GC handle everything (but that is slow).

Original comment by aziz.koe...@gmail.com on 27 Feb 2010 at 5:59

[GoogleCodeExporter]

I guess this should be reported back to Tango.

Original comment by llu...@gmail.com on 27 Feb 2010 at 10:07

[GoogleCodeExporter]

I ended up here looking for (yet another) bug in my GC and noticed you I've 
pointed a bad call to addRange(p, p + size), while is addRange(p, size) really 
(the other signature is used in the low level primitives of the GC). But you 
already tried that and didn't work :-/

Original comment by llu...@gmail.com on 7 Sep 2010 at 2:29