lifting-bits / remill

Library for lifting machine code to LLVM bitcode
Apache License 2.0
1.22k stars 141 forks source link

Fixing the fp80 types windows compilation issue. #592

Open ghost opened 2 years ago

ghost commented 2 years ago

Due to some windows limitation on managing the fp80 types easily , it's better to wrape this two lines to get it compiled properly :

https://github.com/lifting-bits/remill/blob/master/include/remill/Arch/Runtime/Types.h#L101

#if !defined(_WIN32)
    uint8_t padding[sizeof(double) - kEightyBitsInBytes];
#endif

https://github.com/lifting-bits/remill/blob/master/include/remill/Arch/Runtime/Types.h#L82

#if !defined(_WIN32)
  static_assert(10 <= sizeof(native_float80_t), "Invalid `native_float80_t` size.");
#else
  static_assert(8 <= sizeof(double), "Invalid `native_float80_t` size.");
#endif
rileysjc commented 1 year ago

This was very helpful. Few other changes to build with current FP80 type headers (Math.h now).

I got it building with these changes:

union union_ld {
  struct {
    uint8_t data[kEightyBitsInBytes];
    //...
#if (!defined(__CUDACC__) && !defined(_WIN32)) && \
    (defined(__x86_64__) || defined(__i386__) || defined(_M_X86))
    ///...
    uint8_t padding[sizeof(native_float80_t) - kEightyBitsInBytes];
#else
    //...
#endif
  } lds __attribute__((packed));
  native_float80_t ld;
} __attribute__((packed));

//Flip bitfield (need to test with various win32 compilers to see different ABI(s) play nice with alignment and packing variations.

union nan80_t {
  float80_t d;
  struct {
    uint64_t interger_bit : 1;
    uint64_t is_negative : 1;
    uint64_t payload : 62;
    uint16_t is_quiet_nan : 1;
    uint16_t exponent : 15;
  } __attribute__((packed));
} __attribute__((packed));

Development Environment

Using LLVM_v142 with MSVC 2019. This guy provides a clean install and integration for all recent MSVC versions. https://github.com/zufuliu/llvm-utils

Have to use clang-cl for C/C++ compiler to support combination of MSFT/Clang argument syntax.

Also llvm linker encounters issues with unsupported arguments and varying syntax so let Cmake default to MSVC 2019 link.exe. This is working for me:

In CMakeLists.txt: ...

set(CMAKE_GENERATOR "Visual Studio 16 2019" CACHE STRING "")
set(CMAKE_VS_PLATFORM_TOOLSET "LLVM_v142" CACHE STRING "")

...

In LLVM.Common.Props from llvm-utils install in MSVC platform targets directory:

...
  <PropertyGroup>
    <OfficialLLVMInstallDir>D:\Dev\Tools\LLVM</OfficialLLVMInstallDir>
    <OfficialLLVMInstallDir Condition="'$(OfficialLLVMInstallDir)' == ''">$(Registry:HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\LLVM\LLVM)</OfficialLLVMInstallDir>
    <LLVMInstallDir Condition="Exists('$(OfficialLLVMInstallDir)\bin\clang-cl.exe')">$(OfficialLLVMInstallDir)</LLVMInstallDir>
    <ClangClExecutable>$(LLVMInstallDir)\bin\clang-cl.exe</ClangClExecutable>
    <LldLinkExecutable>$(LLVMInstallDir)\bin\lld-link.exe</LldLinkExecutable>
    <LlvmLibExecutable>$(LLVMInstallDir)\bin\llvm-lib.exe</LlvmLibExecutable>
    <LlvmRcExecutable>$(LLVMInstallDir)\bin\llvm-rc.exe</LlvmRcExecutable>
    <UseClangCl>true</UseClangCl>
    <UseLldLink>false</UseLldLink>
    <UseLlvmLib>true</UseLlvmLib>
    <UseLlvmRc>true</UseLlvmRc>
  </PropertyGroup>
...

Lastly, if you're finding yourself with random compile failures then successful compiles, or corrupted intermediate object stores... There are some highly annoying and esoteric llvm bugs when doing incremental linking or mp compiling while also mixing target and host architectures. Since vcpkg/cmake can default to x86 when you're targeting x64 this will drive you mad. If possible do something like: ...-T host=x64 -A x64...

pgoodman commented 1 year ago

@rileysjc can you turn this into a PR?

rileysjc commented 1 year ago

Can do. I'm just about done with a full MSVC 2019 Solution for Remill that folks can use to build the project with relative ease in Windows land. I've also found some cleaner solutions for some of these issues, and others, that I've rolled into the solution. I'll get going on a PR and maybe side discussions for general purpose fixes and workaround as soon as I wrap up my current line items.

trustednstaller commented 1 year ago

Can do. I'm just about done with a full MSVC 2019 Solution for Remill that folks can use to build the project with relative ease in Windows land. I've also found some cleaner solutions for some of these issues, and others, that I've rolled into the solution. I'll get going on a PR and maybe side discussions for general purpose fixes and workaround as soon as I wrap up my current line items.

Any updates regarding this, if you're still willing to share? Some comments in this thread would be very welcomed until you have the time to PR this

pgoodman commented 1 year ago

Ping @rileysjc.

rileysjc commented 1 year ago

Holidays and starting some new projects ate up some cycles late December. Making solid progress on a full MSVC17 WIN AMD64 package that includes a variety of ports (latest version or close to), optimizations (e.g. step LTO, lockless heaps, static wpo, ucrt, external llvm project integrations, etc... I'm on the last leg now, in a perfect world. We can start a sidebar chat with regard to deliverables for the REMILL project. I'll check my patches against current code this week.

pgarba commented 1 year ago

Clang on Windows does support double-80 since some time now (https://reviews.llvm.org/D115441?id=393120)

For bitcode compilation add "-mlong-double-80" to the command line and for clang-cl add "-Xclang -mlong-double-80".

(https://github.com/pgarba/remill/commit/e7298cee430440f08b0d790ae90f98e6480a8179)