Cross-compiling the resultant llvm IR

[DCNick3]

Currently remill-lift produces an IR with target triple (and data layout) set to match the source IR. As far as I can see this is probagated from compilation of the semantics.

I want to compile this IR for a different platform (namely x86 -> aarch64). As far as I can see, remill delegates all platform-dependent work to intrinsics and does not issue int to pointer conversion or any other thing that would be unportable, but I am not entirely sure. Is it possible to make it work? Is just changing the target triple of generated code enough for this?

[pgoodman]

It is definitely possible to make it work; I've previously made an no-longer made emulator that could JIT and run lifted aarch64/x86/amd64 on amd64. The key with Remill is that there are no pointers in the bitcode (other than State * and Memory *). All machine pointers are represented as integers. A tool like Anvill or McSema converts integers to pointers and vice versa. So the tl;dr is that you can lift the bitcode, then can compile it to a new target (effectively ignoring the triple/data layout and going for it). In practice this will work out, due to the way that State structures are designed (using fixed-size types, ensuring no padding in-between structure elements, and avoiding arch-specific types like long double).

[DCNick3]

Yeah, this is what I've been thinking. Thanks for the clarification.

[DCNick3]

Now that I look at it, it seems that the compiled semantics have (at least at one place) a platform dependency: floating point exception macros are implementation defined values, but they are included from <cfenv> in Arch/Runtime/Float.h and used further in the semantics. This makes values passed to __remill_fpu_exception_test_and_clear platform dependent. And it's not only formally platform dependent, running this code snippet

#include <fenv.h>
#include <stdio.h>
int main() {
    printf("FE_DIVBYZERO = %d\n"
           "FE_INEXACT = %d\n"
           "FE_INVALID = %d\n",
            FE_DIVBYZERO, FE_INEXACT, FE_INVALID);
    return 0;
}

On x86_64-pc-linux-gnu gives 4, 32 and 1, while on aarch64-linux-gnu it's 2, 16 and 1 respectively. Seems like a compatibility level with well-defined bit values for those macros is needed.

[pgoodman]

In an msp430 branch I have some compatibility defines for when these macros are absent, but you're right that going and finding what they should be for each type of architecture and then hard-coding that in would be nifty.

The bigger question, though, is to what extent do the values leak? Here's how the intrinsics are used:

https://github.com/lifting-bits/remill/blob/3808e9951db8a7332f361d802de5815cb2877571/lib/Arch/AArch64/Semantics/FLAGS.cpp#L150-L170

• We clear the status flags on the host arch
• Then we do some operation
• Then we ask the FPU of the host arch on which the (compiled, presumably) bitcode is executing: what are the status flags, as you understand them
• Then use the host-specific values to marshal to the emulated status flags

I think as long as host-specific values are used consistently, and don't leak into the emulated program, except in terms of being compared agaisnt other host values for the sake of deriving target values, then we're generally OK.

So, then the follow-up is this, where I think the bigger risk is, and which would be resolved if we followed your advice. The bitcode isn't necessarily produced in a way where the C macros in the bitcode will have consistent values to what those macros are in the target platform. In this case, we definitely want to have consistent values, and we can use this code from the msp430 branch:

https://github.com/lifting-bits/remill/blob/msp430/include/remill/Arch/Runtime/Math.h#L71-L89 (except not do ifndefs)

https://github.com/lifting-bits/remill/blob/msp430/include/remill/Arch/Runtime/Math.h#L124-L160

[DCNick3]

The bitcode isn't necessarily produced in a way where the C macros in the bitcode will have consistent values to what those macros are in the target platform

Well, this would happen when one would want to cross-compile the bitcode to other platform.

For example, one could lift the code with remill built on x86_64 and then use aarch64 cross-compiler to make the lifted bitcode run on aarch64. Then remill runtime would have to use x86_64 values, as semantics were compiled on x86_64 with x86_64 libc. But this runtime must run on aarch64, so aarch64 compiler would be used with aarch64 libc and macro values. Inconsistency here.

I think that this can be solved by introducing shims that are compiled with the same compiler as remill semantics, but, probably, using the platform-independent values from the very beginning would be a better solution to the problem.

The msp430 branch code looks to do the step in the direction of libc independence, that's cool