Call fixup for pointers to member functions

[Nemoumbra]

It seems that the Metrowerks Codewarrior uses a special helper function to perform calls made via the pointers to member functions (ptmfs). Basically, a single ptmf is represented by a struct with sizeof == 12 bytes where all needed info is stored. The semantics of the call are: 1) Load the ptmf struct into 3 fpu regs (4+4+4 == 12) 2) Push it to the stack 3) Setup the this pointer 4) Put the PTMF's stack address to the t9 reg 5) Call __ptmf_scall __ptmf_scall fetches the ptmf from the stack via t9, grabs the function address and unconditionally jumps there. (It doesn't matter where the ptmf comes from - either grabbed from .data or constructed in runtime)

Right now I'm not happy with the decompilation results as Ghidra doesn't show that's a function member call, doesn't show the ptmf setup and doesn't show the arguments. I would like to fix this if possible. Inlining didn't work (the callee got batchered) so I'm currently hoping a call fixup can be applied. Can I make a custom calling convention / call fixup that would force Ghidra to properly detect the ptmf stack setup as described in the list above? If so, where can I look for examples? I personally want Ghidra to show that's a ptmf call somehow and apply the data types for the args and the return value. res = mw_ptmf( /*some deduced ptmf info*/ , args); instead of res = __ptmf_scall(auStack44) (and it's the only place where auStack44 is used).

[LukeSerne]

This could be modelled by making __ptmf_scall have a variable number of arguments, and a PTMF structure pointer. However, to do that, you'd need to use "Custom Storage" to set the PTMF pointer to register t9. Unfortunately, using both "varargs" and "Custom Storage" on the same function confuses the decompiler (see #4334, which has been in triage for over a year now :cry:).

The specific data types for the arguments would depend on the PTMF being executed, right? So making a custom calling convention that hardcodes the data types would probably not be a good solution. You should be able to create a custom calling convention that takes the first argument in t9 and then the remaining arguments "as usual". This is done in the .cspec file. I couldn't find any documentation in the code, but going by this comment, it seems it's as easy as copying the default calling convention, giving it a new name and inserting lines similar to this directly after the <input> tag:

  <pentry minsize="1" maxsize="8">
      <register name="t9"/>
  </pentry>

If you do this, you shouldn't need "Custom Storage", and can probably just set the __ptmf_scall function to use your new calling convention, and set it to varargs.

[astrelsky]

Can I make a custom calling convention / call fixup that would force Ghidra to properly detect the ptmf stack setup as described in the list above? If so, where can I look for examples? I personally want Ghidra to show that's a ptmf call somehow and apply the data types for the args and the return value.

Yes. In the code browser go to edit -> Options for Program Name->Specification Extensions. There you can add your own call fixups.

You can export an existing one as an example. You'll probably want to reformat it though. Note that if the body is multiple lines you may need to use <![CDATA[fixup body]]>

I use these all the time for functions such as __addvdi3 so that the decompiler just shows c = a + b; instead of c = __addvdi3(a,b);

[Wall-AF]

... You can export an existing one as an example. You'll probably want to reformat it though. Note that if the body is multiple lines you may need to use <![CDATA[fixup body]]> ...

Exporting an existing calling convention doesn't give register names etc. in the xml, it just spits out numeric values!

[astrelsky]

... You can export an existing one as an example. You'll probably want to reformat it though. Note that if the body is multiple lines you may need to use <![CDATA[fixup body]]> ...

Exporting an existing calling convention doesn't give register names etc. in the xml, it just spits out numeric values!

I meant for a call fixups. I didn't realize it was asking about both. The exporting of call fixups works at least.

[Nemoumbra]

@LukeSerne Before making this issue I used the void __thiscall __ptmf_scall(void *this,...) signature with varargs and now decided to experiment. void __stdcall __ptmf_scall(void *this,PTMF *ptmf) with the custom storage for the second argument works for the ptmfs that accept no args. As you said, the changes got reverted after I added the ellipsis:

Unfortunately, using both "varargs" and "Custom Storage" on the same function confuses the decompiler

So yeah, confirmed that...

But you also said I should try a custom calling convention, so my question is... If I "hardcode" this in a, say, __mw_ptmfcall convention, then apply it to __ptmf_scall and make it variadic... Would it "unconfuse" the decompiler? (Because if it doesn't, then why bother?)

[astrelsky]

@LukeSerne Before making this issue I used the void __thiscall __ptmf_scall(void *this,...) signature with varargs and now decided to experiment. void __stdcall __ptmf_scall(void *this,PTMF *ptmf) with the custom storage for the second argument works for the ptmfs that accept no args. As you said, the changes got reverted after I added the ellipsis:

Unfortunately, using both "varargs" and "Custom Storage" on the same function confuses the decompiler

So yeah, confirmed that...

But you also said I should try a custom calling convention, so my question is... If I "hardcode" this in a, say, __mw_ptmfcall convention, then apply it to __ptmf_scall and make it variadic... Would it "unconfuse" the decompiler? (Because if it doesn't, then why bother?)

Use a call fixup. Load the function pointer and make it a call. It should decompile to what you are looking for.

__sdivsi3.xml.txt __aeabi_uldivmod.xml.txt

NOTE: I make no guarantees to the accuracy of the attached examples.

[Nemoumbra]

Load the function pointer and make it a call

But this is exactly what __ptmf_scall is doing. Wouldn't I just reimplement inlining? (And we know inlining didn't work)

[astrelsky]

Load the function pointer and make it a call

But this is exactly what __ptmf_scall is doing. Wouldn't I just reimplement inlining? (And we know inlining didn't work)

No it is not. Your call fixup should just implement the call.

t9 = fn_ptr;
call [t9];

[Nemoumbra]

It's not that easy. First of all, I have just noticed that I didn't tell that __ptmf_scall also adjusts the this argument to properly deal with virtual functions. Second of all...yeah... virtual functions and non-virtual functions are handled differently. The whole point of it being a compiler-generated helper function is that it's not trivial!

I'm from my phone right now, but afaik the meaning of __ptmf_scall is: If vtable offset is -1, it's a non-virtual call and we can grab the address from the last 4 bytes of the struct. Else we increment this by the value in the first 4 bytes, grab the vtable offset and take the address from there.

This is why I don't understand how inlining would be any different from an accurate call fixup...

[astrelsky]

It's not that easy. First of all, I have just noticed that I didn't tell that __ptmf_scall also adjusts the this argument to properly deal with virtual functions. Second of all...yeah... virtual functions and non-virtual functions are handled differently. The whole point of it being a compiler-generated helper function is that it's not trivial!

I'm from my phone right now, but afaik the meaning of __ptmf_scall is: If vtable offset is -1, it's a non-virtual call and we can grab the address from the last 4 bytes of the struct. Else we increment this by the value in the first 4 bytes, grab the vtable offset and take the address from there.

This is why I don't understand how inlining would be any different from an accurate call fixup...

You don't need the real address. The goal is to show a function call without the noise. Take the pointer to member function, move it to t9 and then call [t9]. The result will be an indirect function call without the implementation details, which is what you want.

If you take a software floating point add function and inline it, you're going to have a bad day. However, if you use a call fixup of regC = regA f+ regB; you're just going to see floating point addition in the decompiler. How it actually does the floating point arithmetic doesn't matter unless you need emulation.

[Nemoumbra]

You don't need the real address.

If we follow your logic, I also don't want the real addresses for normal calls (and don't want proper decompilation).

Real life example of a virtual ptmf values: {0x0, 0xA8, 0xC}. Be my guest and explain how I would benefit from a call to 0xC? The vtable start is at *(this + 0x0 + 0xC) and the function I need is at *(vtable_start + 0xA8).

Real life example of a nonvirtual ptmf values: {0x0, -1, 0x08837960}. Here the address is 0x08837960, because the second value is negative.