Compiling x86 operations with x64 target and vice versa

[Hypercall]

When using x86 code with x64 it seems to not compile it correctly. I want to be able to set x64 register with a value of size uint64_t with x86 target. It is needed to break through the heavens gate to inject dlls from x32 to 64 bit processes.

Perhaps a simple macro would be enough (like #define XBYAK32_64), so you would use your own size and add support for 64 bit operands in x32 mode.

[herumi]

How about https://github.com/herumi/xbyak/tree/XBYAK64_on_32bit with a macro -DXBYAK64 on 32-bit?

[Hypercall]

Looks good. However, it throws ERR_BAD_COMBINATION when trying to push a x64 register. I fixed by checking for bit == 32 in function opPushPop. Could you add it?

[herumi]

On my environment such as Windows 32-bit Visual Studio and g++ -m32 on Linux, the following code runs well. Could you tell me what OS/compiler and what code do you use?

#define XBYAK64
#include <xbyak/xbyak.h>

struct Code : Xbyak::CodeGenerator {
    Code()
    {
        push(rax); // 0x50
        pop(rax); // 0x58
    }
};

[Hypercall]

I am using Windows 10 x64 (build 2004) with MSVC (Visual Studio 2019, latest std features) . It throws the exception when trying to run the function "get_code" with the following sample:

[herumi]

Hum, it is strange. I run the following code on Windows 10 Pro + Visual Studio 2019 and it seems to run well. Could you try it?

// t.cpp
#define XBYAK64
#include <xbyak/xbyak.h>

struct Code : Xbyak::CodeGenerator {
    Code() : Xbyak::CodeGenerator(4096)
    {
        mov(rax, 0x1234567890123456ull);
        push(rax);
        pop(rax);
        ret();
        dump();// show 48B856341290785634125058C3
    }
};

int main()
    try
{
    printf("sizeof(void*)=%zd\n", sizeof(void*));
    Code c;
    auto f = c.getCode<size_t (*)()>();
#ifdef _WIN64
    printf("call f()=%zx\n", f());
#endif
} catch (std::exception& e) {
    printf("err %s\n", e.what());
}

git clone https://github.com/herumi/xbyak -b XBYAK64_on_32bit
// copy t.cpp in xbyak directory

open x64 Native Tools

xbyak>cl /EHsc t.cpp -I ./ && t.exe
Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28806 for x64
t.cpp
Microsoft (R) Incremental Linker Version 14.26.28806.0

/out:t.exe
t.obj
sizeof(void*)=8
48B856341290785634125058C3
f=00000277BA602000
call f()=1234567890123456

Open x86 Native Tools Command Prompt for VS 2019.

xbyak>cl t.cpp /EHsc -I ./ && t.exe
Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28806 for x86

t.cpp
Microsoft (R) Incremental Linker Version 14.26.28806.0

/out:t.exe
t.obj
sizeof(void*)=4
48B856341290785634125058C3
f=01545000

[Hypercall]

This seems to work fine. I am not sure why it did not work before, I guess I cloned the wrong version. It still throws the exception "ERR_BAD_COMBINATION" when trying to push a 32 bit register (eax for example) with the macro XBYAK64 defined (target x86):

#define XBYAK64
#include "../xbyak/xbyak.h"

struct Code : Xbyak::CodeGenerator {
    Code() : Xbyak::CodeGenerator(4096)
    {
        push(eax);
        mov(eax, 0);
        pop(eax);
        ret();
        dump();
    }
};

int main()
try
{
    printf("sizeof(void*)=%zd\n", sizeof(void*));
    Code c;
    auto f = c.getCode(); // throws here
#ifdef _WIN64
    //printf("call f()=%zx\n", f());
#endif
}
catch (std::exception& e) {
    printf("err %s\n", e.what());
}

Edit: It also seems that the 32 bit operations like pushad or popad are undefined when defining the macro XBYAK64

[herumi]

It still throws the exception "ERR_BAD_COMBINATION" when trying to push a 32 bit register (eax for example) with the macro XBYAK64 defined (target x86):

It also seems that the 32 bit operations like pushad or popad are undefined when defining the macro XBYAK64

push(r32), pop(r32), pushad and popad are invalid on 64-bit mode. cf. https://www.felixcloutier.com/x86/push https://www.felixcloutier.com/x86/pop https://www.felixcloutier.com/x86/pusha:pushad https://www.felixcloutier.com/x86/popa:popad

[Hypercall]

It still throws the exception "ERR_BAD_COMBINATION" when trying to push a 32 bit register (eax for example) with the macro XBYAK64 defined (target x86):

It also seems that the 32 bit operations like pushad or popad are undefined when defining the macro XBYAK64

push(r32), pop(r32), pushad and popad are invalid on 64-bit mode. cf. https://www.felixcloutier.com/x86/push https://www.felixcloutier.com/x86/pop https://www.felixcloutier.com/x86/pusha:pushad https://www.felixcloutier.com/x86/popa:popad

I want to be able to create shellcodes to hijack threads for both 32 and 64-bit processes. For this both modes are basically required. I am aware that this is not suppported on 64-bit mode. A mix of both modes like the sample below would be great.

class sample32: Xbyak::CodeGenerator {
    public:

        sample32() {
            pushad();
            //... some hijacking stuff, call a func...
            popad();
            //push(xxxxxxxxxxxx);
            ret;
        }

};

class sample64: Xbyak::CodeGenerator {
    public:

        sample64() {
            // push all registers manually ...
            push(r8);
            // call some func...
            // pop em all...
            // jmp back

        }

};

[herumi]

I want to be able to create shellcodes to hijack threads for both 32 and 64-bit processes.

The current xbyak is not designed to generate mixing encoding. So could you try to split the code as the following way?

// main.cpp
#include <stdint.h>

extern void genCode32bit(uint8_t *code, size_t codeSize);
extern void genCode64bit(uint8_t *code, size_t codeSize);

struct code {
    alignas(4096) uint8_t code[4096];
    void genCode(bool is32bit)
    {
        if (is32bit) {
            genCode32bit(code, sizeof(code));
        } else {
            genCode64bit(code, sizeof(code));
        }
    }
};

// code32.cpp
#define XBYAK32
#include <xbyak/xbyak.h>

struct Code32 : Xbyak::CodeGenerator {
    Code32(uint8_t *code, size_t codeSize)
        : Xbyak::CodeGenerator(codeSize, code)
    {
        mov(eax, 0);
        push(eax);
        pop(eax);
        ret();
    }
};

void genCode32bit(uint8_t *code, size_t codeSize)
{
    Code32 code32(code, codeSize);
}

// code64.cpp
#define XBYAK64
#include <xbyak/xbyak.h>

struct Code64 : Xbyak::CodeGenerator {
    Code64(uint8_t *code, size_t codeSize)
        : Xbyak::CodeGenerator(codeSize, code)
    {
        mov(rax, 0);
        push(rax);
        pop(rax);
        ret();
    }
};

void genCode64bit(uint8_t *code, size_t codeSize)
{
    Code64 code64(code, codeSize);
}

[Hypercall]

Seems to work fine!