orangeduck / tgc

A Tiny Garbage Collector for C
Other
968 stars 64 forks source link

Problems with C++ virtual destructor #7

Open cia48621793 opened 7 years ago

cia48621793 commented 7 years ago

So I'm trying to exploit tgc for C++ automatic GC dream:

class object {
    public: object();
    public: virtual ~object() {}
};
void* __cdecl operator new(unsigned int size) throw(std::bad_alloc) {
    tgc_run(&gc);
    return tgc_alloc(&gc, size);
}

void* __cdecl operator new[](unsigned int size) throw(std::bad_alloc)  {
    tgc_run(&gc);
    return tgc_calloc(&gc, 1, size);
}

void __cdecl operator delete(void *mem) throw()  {
    tgc_run(&gc);
    return tgc_free(&gc, ptr);
}

object::object() {
    auto dtor = (void(*)(void*))**(void***)this;
    tgc_set_dtor(&gc, this, dtor);
}

The dtor is virtualized. However, when the program is sweeping and deleting the object, the dtor was not able to be called, and instead a segfault occurred. Can someone figure out what's happening?

orangeduck commented 7 years ago

Hi,

What is the line (void(*)(void*))**(void***)this meant to be doing? It looks like you are casting this to a function pointer and then passing it to tgc to be the destructor - but this is just a pointer to the object is it not? It needs to be a C function pointer.

Unfortunately it seems in C++ you can't get a function pointer to a C++ class destructor:

http://stackoverflow.com/questions/10858998/how-do-i-get-the-member-function-pointer-of-a-destructor

So I am not sure you can use tgc in this way for C++.

I think it may still be possible to adapt tgc for use with C++ though. One idea would be something along these lines:

  1. Remove tgc_calloc and tgc_realloc
  2. Replace all calls in tgc to free with del
  3. Replace all calls in tgc to malloc with new (you will have to use a template to pass the type through)
  4. Overload new, new[], del, del[] to call tgc_alloc/tgc_run and tgc_free/tgc_run.

There are probably still some problems E.G. that we want to make sure tgc calls the original versions of new and del instead of the overloaded versions. I'm not enough of a C++ wizard to work this one out.

cia48621793 commented 7 years ago

@orangeduck Hey, This:

(void(*)(void*))**(void***)this

Is actually accessing the virtual dispatcher table on this object. Basically, it looks like this hierarchy:

this
-> vtable_ptr
   -> vtable
      -> vdtor (if available)/vfn 0
      -> vfn 1
      -> vfn 2
      -> ...etc

So what I was actually doing is this->vtable_ptr->vtable->vdtor. Look at this article for a better explanation. I can get the virtual destructor correctly, the debugger recognized the address of dtor in tgc as deleting destructor, however it just couldn't get called for some spooky reasons...access violation? Was it not in the .code section? A little update: The virtual destructor hack works on x64 platforms but not x86. Possible reasons are unknown, maybe due to memory alignment problem.

orangeduck commented 7 years ago

Nice - never knew about that hack.

One other thing to be careful about is that tgc does not order destructors. E.G when an object goes out of scope so does all of it's children - tgc doesn't resolve the ordering of destructors in any way and some of the child objects may have their destructors called before the parent or vise versa.

Otherwise I am interested to see how this goes 👍