search

discoproject / discodb

An efficient, immutable, persistent mapping object
http://discodb.readthedocs.org/
BSD 3-Clause "New" or "Revised" License
99 stars 31 forks source link

Malloc assertion failure when 65k DiscoDB objects go out of scope #1

Open mittonk opened 12 years ago

mittonk commented 12 years ago

Using Python 2.6 and discodb 0.2 or 0.1:

import discodb

data = {'foo': 'bar'}
arr = []
for x in xrange(65537):
    print x
    data_disco = discodb.DiscoDB(data)
    arr.append(data_disco)
print "Got to the end safely."
1
...
65535
65536
Got to the end safely.
python: malloc.c:3551: munmap_chunk: Assertion `ret == 0' failed.
Aborted

Smaller collections of DiscoDB objects work as expected.

tuulos commented 11 years ago

Thanks @mittonk for reporting the error - and sorry that it took this long for me to reply!

I can reproduce the error. It is most likely caused by the process exceeding the maximum number of mmaps, as defined in /proc/sys/vm/max_map_count, which at least on my machine is 65530.

Unfortunately the problem seems to be in realloc() which doesn't return NULL although it should in this situtation, see:

http://sourceware.org/bugzilla/show_bug.cgi?id=13276

Hence the bug doesn't seem to be in DiscoDB per se. The case is pretty extreme, so I guess there's not much we can do about it right now.

mittonk commented 11 years ago

Nice investigation, @tuulos . glibc does seem like the right level to approach this, I'm fine with no changes to discodb.