crashappsec / libcon4m

Base Compiler and Runtime Support for con4m
Apache License 2.0
0 stars 0 forks source link

Full memory checking for GC'd heap #78

Closed viega closed 1 week ago

viega commented 1 week ago
  1. I added explicit support for ASAN poisoning, but it was not partic…ularly valuable in any way, so I:
  2. Added a tremendous amount of heap checking.
  3. I used this to find and fix:

Hash value caching moved from the con4m object header into the alloc header. This way, any GC'd pointer will have its hash value cached. Not doing it that way was an oversight. And, as a result, occasionally something would be in a dict or set, but the hash value was based on its OLD pointer value, so a collection would give the same value a new hash. Not many things like this are used as keys right now, but one was the module worklist at the top level, so if a collect happened at the wrong time, you could end up in an infinite loop, because some module was never going to get removed from the set (but could always be retrieved from it). To test this out, I lowered the starting heap size all the way down to 1K to try to trigger the problem as much as possible (tho the heap doubles in size if, after the previous collect, there's deemed not sufficient space). Amazingly it didn't slow things down.

Heap checking (enabled in debug builds) includes:

  1. Full sweep of the heap after collection looking for garbage that might not be garbage.
  2. Guards AFTER each allocation (already existed before).
  3. Checking of guards during collections.
  4. A ring buffer of recent allocs; guards are checked after each allocation for anything in the ring buffer.

Also along the way:

Closes #71