CMake project generation type on Windows

[hozuki]

Current bootstrap script generates one type of VS solution based on %BITNESS% (empty or Win64). This will cause MSBuild always failing to build. An example error message:

mcsema-print-runtime-x86.dir\Release\print_PE_32_windows.obj : fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'x64'

This is because when using "Visual Studio xx.x Win64" configuration, all project platform in .vcxprojs are x64. Then it fails to link against a 32-bit static library, appointed by CMake script.

On 32-bit machines this issue is suppressed because 64-bit build steps are skipped (e.g. here).

[alessandrogario]

Hello Hozuki; please take a look at my branch: alessandro/refactor/common-installer

Contains the source code (WiX Toolset) for an MSI installer, and a working CMake project. There is also a new (crossplatform) bootstrap script written in Python, that will work around the issue you are describing. This works by generating the project a second time in order to build the missing 32-bit parts.

[hozuki]

Thank you. I started to use the python script. Where should the third party packages be placed at? I got an error:

CMake Warning at CMakeLists.txt:33 (find_package):
  Could not find a package configuration file provided by "LLVM" (requested
  version 4.0) with any of the following names:

    LLVMConfig.cmake
    llvm-config.cmake

  Add the installation prefix of "LLVM" to CMAKE_PREFIX_PATH or set
  "LLVM_DIR" to a directory containing one of the above files.  If "LLVM"
  provides a separate development package or SDK, be sure it has been
  installed.

(The full output also contains this kind of paragraphs of version 3.8 and version 3.9.)

"llvm" only appears a few times in Python scripts so it is easy to check: not the script's problem.

[alessandrogario]

Hello Hozuki,

use the --install-deps switch of the python script

[hozuki]

Hello Alessandro,

The script is amazing. After modifying some VS and LLVM detection code it worked perfectly.

I'm not sure how to configure the MSI part, so I built the Python egg as before, and moved all binaries together. After that I tried to invoke mcsema-disass. It ends with a CalledProcessError because IDA returned status 1. Although I specified a --log_file argument, there are still no logs. IDA seems started , .id*, .nam and .til are generated.

I am using IDA 6.8 with IDAPython 1.7.2. The Python environment is 2.7.13 32-bit. IDAPython can be loaded when running in GUI mode. (Seriously, I think the readme should suggest manually installing Python instead of using Chocolatey. The latter always tries to install a 64-bit version on 64-bit Windows. IDAPython does not run under 64-bit Python, for now.)

[alessandrogario]

Hello Hozuki,

you have to install the WiX toolset first from their website http://wixtoolset.org and then pass the --build-msi-package to the installer script. You can also use --help to see the full list of available options.

Not sure about the readme, but i always install Python manually (I don't have Chocolatey). Could you please try to use my GUI script? Just open the IDB project and run mcsema.py. Just select the 'main' entry point and see if it runs.

[hozuki]

Hello Alessandro,

I installed WiX toolset and built the MSI. After clicking the MSI nothing seemed to happen. So I checked the .wxs and all it does seems to be creating shortcuts in Start Menu, which open consoles from the directory of mcsema-disass. Well the MSI may be unnecessary.

---

Then I began to try the GUI script. IDA says "no module named PyQt5". OK now I finally see why mcsema-disass couldn't work. This script is meant for IDA 6.9 which replaces PySide with PyQt 5. With the help from Hex-Rays' Blog, I hacked mcsema.py.

Imports:

try:
    from PyQt5 import QtCore, QtGui, QtWidgets
    USE_NEW_QT = True
except ImportError:
    from PySide import QtCore, QtGui
    QtWidgets = QtGui
    USE_NEW_QT = False

OnCreate():

if USE_NEW_QT:
    self.parent = self.FormToPyQtWidget(form)
else:
    self.parent = self.FormToPySideWidget(form)

Then it says module sip is missing. I googled for a while and sip turns out to be only available for Python 3.x on Windows now, by default. In FormToPySideWidget, sip is not needed. So here:

if USE_NEW_QT:
    import sip

(Before this simple solution, I followed this article to build sip for Python 2.7. Just use nmake instead of make, and the path should be C:\Python27\Lib\site-packages.)

---

Finally the script is running. But it is still broken. E.g. the "add one item" in "Function List" tab isn't working, but "add all" works.

Anyway, I tried to export the IDB project. If all items are selected in the tab pages:

  File "mcsema/build/mcsema-install/share/mcsema/ida_plugin_installer/mcsema_disass/ida\get_cfg.py", line 2509, in export_cfg
    recoverCfg(eps, outf, exports_are_apis)
  File "mcsema/build/mcsema-install/share/mcsema/ida_plugin_installer/mcsema_disass/ida\get_cfg.py", line 1983, in recoverCfg
    ea = idc.LocByName(name)
  File "IDA6.8\python\idc.py", line 1943, in LocByName
    return idaapi.get_name_ea(BADADDR, name)
  File "IDA6.8\python\idaapi.py", line 51097, in get_name_ea
    return _idaapi.get_name_ea(*args)
TypeError: in method 'get_name_ea', argument 2 of type 'char const *'

If no item is selected, it seems to work, and produce what the walkthrough demonstrates:

... (omitted)
  File "mcsema/build/mcsema-install/share/mcsema/ida_plugin_installer/mcsema_disass/ida\get_cfg.py", line 247, in doesNotReturn
    raise Exception("Unknown external: " + fname)
Exception: Unknown external: GetStartupInfoA@4

I will continue to work on the last case.

[alessandrogario]

Hello Hozuki,

The MSI package will copy all required files to the C:\mcsema folder, and is made to be run unattended. It also includes a script to (re)install the McSema component in the system Python environment. You only need this if you want to redistribute the build you generated.

The Python module errors you are experiencing can be fixed using my Python builder located here: https://github.com/trailofbits/mcsema/blob/master/tools/build_idapython.sh

One error that it doesn't fix is the one about PySide; I should probably add some code to test which version of IDA is being used to run the script.

I will check the function list widget (thanks for the bug report!).

The last error can be fixed by using the external definitions file; the default one (windows.txt) should already contain the definition for the GetStartupInfoA function. You can pass this file to McSema by using the --std-defs switch when calling the mcsema-disass script. If you are using the GUI frontend, there's a page dedicated to this functionality. Take a look at the FAQ here: https://github.com/trailofbits/mcsema/blob/master/docs/CommonErrors.md#unknown-external-function-name

[hozuki]

Hello Alessandro,

I think The MSI should require administrator privilege to run because currently it doesn't, and it didn't create anything under C:\. Shortcuts and registry entries are created, though.

EDIT:

Wow it installed to my F drive, not C drive...

[hozuki]

Hello Alessandro,

Thank you. After adding some definitions now the GUI script is able to generate a .mcd file. However mcsema-lift reports:

Failed to deserialize protobuf module

Unable to read module from CFG

Python and C++ are using the same .proto so they should generate the same protobuf contract. String encoding (UTF-8 vs. ANSI on non-English Windows) is not likely because Protobuf always uses UTF-8. I can't figure out the problem here.

[alessandrogario]

Hello Hozuki,

the MSI source code does not define the TARGETDIR variable, meaning that the ROOTDRIVE value is used instead. If you have a Windows folder on that drive, this may be the reason it has been installed there.

WiX source code: https://github.com/trailofbits/mcsema/blob/alessandro/refactor/common-installer/windows/msi_installer/trailofbits_mcsema.wxs Reference: https://msdn.microsoft.com/en-us/library/windows/desktop/aa372064(v=vs.85).aspx

The MSI packages should already request admin privileges automatically when installing.

I will look into that error and let you know if I discover anything; thanks for taking the time to help me test my branch!

[hozuki]

You're welcome. Thanks for this magic repo (although I haven't successfully run the whole procedure yet 😞).

[artemdinaburg]

Closing this issue since mcsema 2 does not yet build on Windows. This issue is relevant to mcsema-legacy