VTable Alignment Issue

[ochafik]

From @asherkin on June 13, 2012 16:37

BridJ appears to automatically handle the "extra" virtual dtor that GCC adds into the vtable (which in most cases would give you seamless comparability).

However, in the case of abstract classes, this is not present, causing the function indices to be miss-aligned when calling.

While I haven't tested (and therefore really shouldn't mention it, but, shhh :P), I'm guessing this'll also be a problem with how MSVC and GCC order overloaded functions (and even weirder, overloads from inherited classes) differently as well.

Copied from original issue: ochafik/nativelibs4java#318

[ochafik]

Hi @asherkin ,

Thanks for your report ! BridJ is in dire need for work on vtable issues, especially when it comes to abstract classes (see issue #287). Any pointers to useful doc and / or contribution will be highly welcome !

Cheers

[ochafik]

From @asherkin on June 20, 2012 1:14

Hi @ochafik,

In terms of BridJ, I think the best course of action would be to extend the Virtual annotation to also specify compiler (MSVC and GCC being the big ones, and the handful of others I tested were ABI compatible with GCC with regards to vtable ordering) along with index.

Would also need a mechanism (another annotation at the top of the class?) to specify the correct compiler per-platform (since libraries might need it specified manually, i.e. GCC via MinGW on Windows), probably defaulting to MSVC on Windows and GCC(-compatible) on Linux / Darwin.

In the short term, this would allow manual fixing of order / position, and in the long term JNAerator could be extended to handle generating the correct indices.

I'm still doing research on how different compilers handle ordering of virtual functions, there are quite a few variables in play. Big issues are:

• Same compiler on different platforms
  • I've only tested recent versions of GCC on Windows (MinGW), Linux and Darwin in this regard, and they all matched my simple test
• Same compiler on different architectures
  • Only tested with programs compiled for 32-bit currently.
• Different versions of the same compiler.
  • Versions of GCC 3.1.1 and prior (and probably a few later) have different ordering.

The two major one's (which I touched on lightly originally, but I've now tested) I've run into using BridJ, and we've run into often at AlliedModders are:

• GCC (since some version later than 3.1.1) orders the vtable in declaration-order, including overloaded functions, while MSVC reverses the order of overloads.
• When there is a virtual destructor present, MSVC takes up the one entry you'd expect, while GCC has 2.
  • BridJ handles this one, but assumes there is always a virtual destructor present.

All my in-depth testing has been with MSVC 10 and GCC 4.7.0, 32-bit, and only single-inheritance up to this point.

While it'll take time to iron out all the edge-cases, getting the infrastructure (the annotation changes) in-place in BridJ would help to work-around it now (instead of multiple compiles), and give a base for extending it in the future.

Hopefully this makes somewhat sense.

Regards, Asher

For reference:

class CTestBase
{
public:
    virtual ~CTestBase() { printf("~CTestBase.\n"); }

public:
    virtual void FunctionA() { printf("Function A.\n"); }
    virtual void FunctionB() { printf("Function B.\n"); }
    virtual void FunctionC() { printf("Function C.\n"); }
    virtual void FunctionC(int a) { printf("Function C int.\n"); }
    virtual void FunctionC(float a) { printf("Function C float.\n"); }
    virtual void FunctionD() { printf("Function D.\n"); }
    virtual void FunctionE() { printf("Function E.\n"); }
};

class CTest: public CTestBase
{
public:
    virtual void FunctionA() { printf("Function 2 A.\n"); }
    virtual void FunctionB() { printf("Function 2 B.\n"); }
    virtual void FunctionC() { printf("Function 2 C.\n"); }
    virtual void FunctionC(int a) { printf("Function 2 C int.\n"); }
    virtual void FunctionC(char a) { printf("Function 2 C char.\n"); }
    virtual void FunctionC(short a) { printf("Function 2 C short.\n"); }
};

MSVC:

// Listing generated from "test.exe" by linux_vtable_dump.ida
// vtable block @ 0x0040819C
0   ~CTestBase
1   Function_2_A
2   Function_2_B
3   Function_C_float
4   Function_2_C_int
5   Function_2_C
6   Function_D
7   Function_E
8   Function_2_C_short
9   Function_2_C_char

GCC:

// Listing generated from "a.exe" by linux_vtable_dump.ida
// vtable block @ 0x004031E8
0   ~CTestBase
1   ~CTestBase
2   Function_2_A
3   Function_2_B
4   Function_2_C
5   Function_2_C_int
6   Function_C_float
7   Function_D
8   Function_E
9   Function_2_C_char
10  Function_2_C_short

[ochafik]

From @asherkin on June 20, 2012 12:20

Looks like I completely failed regarding virtual destructors, the version of BridJ I was testing with was pretty old and this was actually fixed a few months ago (just looking at commits, I don't have access to a Linux machine to test right now).

I stand by trying to patch it up at runtime is not the best course of action though :P.

[ochafik]

From @hardtoe on August 14, 2013 16:2

I wrote a CustomCPPRuntime class (extends CPPRuntime) that works around this issue. I'm not confident in the robustness of my fix however. I check to see if the class looks like it only has virtual methods according to the bindings and that it's a GCC-compiled library. If it is then I add '1' to the virtual table indexes for each virtual method.

To do this I augmented the CPPRuntime.getAbsoluteVirtualIndex() method as follows:

    int getAbsoluteVirtualIndex(
        final Method method, 
        final int virtualIndex, 
        final Class<?> type,
        final NativeLibrary typeLibrary
    ) {
        int virtualIndexCorrection;

        if (!typeLibrary.isMSVC() && isPureAbstractClass(type)) {
            virtualIndexCorrection = 1;

        } else {
            virtualIndexCorrection = 0;
        }

        return super.getAbsoluteVirtualIndex(method, virtualIndex + virtualIndexCorrection, type);
    }

    private boolean isPureAbstractClass(Class<?> type) {
        for (final Method m : type.getMethods()) {
            if (
                Modifier.isNative(m.getModifiers()) && 
                m.getAnnotation(Virtual.class) == null &&
                !Object.class.isAssignableFrom(m.getDeclaringClass())
            ) {
                return false;
            }
        }

        return true;
    }