search

manuel-serrano / bigloo

a practical Scheme compiler
http://www-sop.inria.fr/indes/fp/Bigloo
Other
133 stars 20 forks source link

GC crashes reporting negative heap expansion #83

Closed svenha closed 1 year ago

svenha commented 1 year ago

I have a complex 64bit application that consistently crashes as follows (4.4c and current 4.5a beta):

GC Warning: Failed to expand heap by -2113925120 bytes
GC Warning: Failed to expand heap by -2147479552 bytes
GC Warning: Out of Memory! Heap size: 2105 MiB. Returning NULL!
*** ERROR:bigloo: `segmentation violation' exception -- raised

The machine has 64 GB RAM and the application uses only some of them. It looks like the heap size is going to be doubled, but this byte count overflows as a positive long integer? Just a wild guess based on the above lines.

lambda-calcul commented 1 year ago

Hi Sven,

Do you have an idea of what's going on?

I have already observed similar problems. They were due to wrong data structure expansions. For instance, the hash table implementation used to double the bucket vector when the load of the table became too high. This was causing this sort of problems.

One way to detect this situation, might be try to have a look with gdb. Compile your program with "-cg" and run it under gdb. When the program crashes, inspect the stack and try to find out which was the last C function invoked. This might tell us which Scheme or C function is causing the memory explosion. If this does not work, I suggest that you contact me privately so that we could more easily inspect your code and conduct some experiments on your actual program.

svenha commented 1 year ago

Thanks for the feedback. I am investigating ...

At least, the warning messages could be improved by changing lines like

"Failed to expand heap by "

to avoid negative numbers. But this comes form the BDW garbage collector, I guess. I opened an issue here: https://github.com/ivmai/bdwgc/issues/496

lambda-calcul commented 1 year ago

Yes. You are correct. Good idea to mention that directly to Ivan (the GC maintainer).

svenha commented 1 year ago

The strange warning message made it hard to understand the problem. The cause of the problem is trivial: a string became way too large. The warning message will be fixed upstream in bdwgc, I hope. So, I am closing the issue for bigloo.