Write a tool to ease with embedded allocator debugging

[Manu343726]

The stateful allocator model introduced by EmbeddedAllocator (in short words "place all allocator metadata in the arena itself") makes easy to manage stateful allocators, but the allocator state is no longer visible with a debugger.

For windows there's a good solution: Visual Studio provides an API for extending debugging visualizers: https://msdn.microsoft.com/en-us/library/jj620914.aspx

Pinging @foonathan just for the case... :P

[foonathan]

I have no experience with debuggers, I rarely ever use them.

You could pack the metadata in a struct and view the struct in the debugger but otherwise... Write correct code? :D

[Manu343726]

I also try to write correct code so I don't have to leave my vim-zen-cave, but there are situations where this is not possible xD. Sooo you wrote a high quality allocator library with no debugging at all???? LOL

[foonathan]

I have debugged some parts of my freelists implementation but since the most difficult one is an XOR linked list, I had a similar issue.

I mostly do printf-debugging, though, I find it much easier.

[Manu343726]

Just for documentation sake, this is what I mean by "ease to debug embedded allocator": This is a capture of the reflection example, specifically member function invocation:

Yeah, it's the same shot from twitter....

The reflection engine has one freelist-based pool per object type so allocations are "optimized". The highlighted lines in the memory viewer are the beginning of the int object pool. The structure of the EmbeddedAllocator storage is as follows:

+-------------------------------+ <--- arena begin
|    pointer to end of arena    |
+-------------------------------+
| alloc metadata length (bytes) | 
+-------------------------------+ <--- metadata begin
|                               | ^
|             ...               | | metadata
|                               | v
+-------------------------------+ <--- storage begin
|                               | ^
|             ...               | | data
|                               | v
+-------------------------------+ <--- storage/arena end

so for the FreeList allocator the arena looks like this:

+-------------------------------+ <--- arena begin
|    pointer to end of arena    |
+-------------------------------+
|   4/8 bytes (sizeof(void*))   | 
+-------------------------------+ <--- metadata begin
|      pointer to head node     |
+-------------------------------+ <--- storage begin
|                               | ^
|             ...               | | data
|                               | v
+-------------------------------+ <--- storage/arena end

In the screenshot, the int object arena (At least the first block, there's really a chain of fixed-size freelists) goes from addresses 0x01175260 to 0x01176260, with the result object from function invocation at address 0x01175270, and the head (@0x01175268) pointing to node at 0x0117526C (Node used previously by the function argument object).

[foonathan]

Put that layout into a struct instead of working with offsets and casts. Then place the struct in the memory.

Look at the chunk handling here: https://github.com/foonathan/memory/blob/master/src/detail/small_free_list.cpp

[Manu343726]

I wanted to avoid padding, but maybe you are right. Placement new is our friend...

[Manu343726]

Here's a tutorial about writing debug visualizers for GDB: http://plohrmann.blogspot.com.es/2013/10/writing-debug-visualizers-for-gdb.html

I don't do python since biicode closed, let's see if I haven't lost my pythonic spirit...

[Manu343726]

GDB Python API for custom pretty printers: http://sourceware.org/gdb/onlinedocs/gdb/Pretty-Printing-API.html#Pretty-Printing-API

Also in the same webpage, a mini-tutorial: http://sourceware.org/gdb/onlinedocs/gdb/Writing-a-Pretty_002dPrinter.html#Writing-a-Pretty_002dPrinter

[Manu343726]

Whenever I get time to solve all the other ugly stuff (no tests, no docs, etc), opening a new issue about migrating to foonathan/memory would be a great idea.