search

DFIR-ORC / dfir-orc

Forensics artefact collection tool for systems running Microsoft Windows
https://dfir-orc.github.io
GNU Lesser General Public License v2.1
368 stars 42 forks source link

Multiples issues on compilation with VS2019 and VS2022 #64

Closed NicknameNotTaken closed 2 years ago

NicknameNotTaken commented 2 years ago

Hi,

I tried compiling dfir-orc by following the Readme, but still got issues.

Here is what i did, from a Windows 10 (Microsoft Windows [version 10.0.19044.1645]):

From here, several issues:

  1. The libwinpthread can't be found:

    [DEBUG] Downloading https://repo.msys2.org/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst
[...]
[DEBUG] Downloading https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst
Error: Failed to download from mirror set:
https://repo.msys2.org/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: failed: status code 404
https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: failed: status code 404
[...]
https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: failed: status code 404
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: WinHttpSendRequest() failed: 12007
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: WinHttpSendRequest() failed: 12175
[...]
https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst: failed: status code 404

  2. If i switch to the branch "master" of vcpkg and regenerate it (bootstrap-vcpkg.bat), the packages are found and installed. But then, the exact same issue as #61 appears.

  3. Trying my luck, i removed this dependency in the project and this step finally succeeds (reminder: cmake -G "Visual Studio 16 2019" -A Win32 ..):

    [...]
-- Using toolchain: C:/Users/forensics/source/repos/dfir-orc/external/vcpkg\scripts\buildsystems\vcpkg.cmake
-- Using vcpkg triplet: x86-windows-static
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.1.
-- The C compiler identification is MSVC 19.29.30143.0
-- The CXX compiler identification is MSVC 19.29.30143.0
-- The ASM_MASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x86/ml.exe
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x86/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x86/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Boost: C:/Users/forensics/source/repos/dfir-orc/external/vcpkg/installed/x86-windows-static/include (found version "1.79.0")
-- Found VisualStudio: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community
-- RapidJSON found. Headers: C:/Users/forensics/source/repos/dfir-orc/external/vcpkg/installed/x86-windows-static/share/rapidjson/../../include
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/forensics/source/repos/dfir-orc/build-x86

  4. Onto the next command then: cmake --build . --config MinSizeRel -- -maxcpucount. But here, i get a TON of build errors... Example for OrcLib: 

    Generating Code...
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(69,26): error C2664: 'OutputIt fmt::v8::format_to<std::back_insert_iterator<fmt::v8::basic_memory_buffer<wchar_t,500,std::allocator<wchar_t>>>,,0>(OutputIt,fmt::v8::basic_
format_string<char>)': cannot convert argument 2 from 'const wchar_t [27]' to 'fmt::v8::basic_format_string<char>' [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          OutputIt=std::back_insert_iterator<fmt::v8::basic_memory_buffer<wchar_t,500,std::allocator<wchar_t>>>
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(51,5): message : No constructor could take the source type, or constructor overload resolution was ambiguous (compiling source file C:\Users\forensics\source\repos\dfir-or
c\src\OrcLib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\external\vcpkg\installed\x86-windows-static\include\fmt\core.h(3146,17): message : see declaration of 'fmt::v8::format_to' (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcL
ib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(86): message : see reference to function template instantiation 'void Orc::Log::Logger::Log<FacilityIt,std::chrono::system_clock::time_point,const wchar_t(&)[27],>(Facilit
yIt,FacilityIt,const Timepoint &,Orc::Log::Level,Arg0)' being compiled [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          FacilityIt=std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_ptr<Orc::Log::SpdlogLogger>>>>,
          Timepoint=std::chrono::system_clock::time_point,
          Arg0=const wchar_t (&)[27]
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(188): message : see reference to function template instantiation 'void Orc::Log::Logger::Log<std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_p
tr<Orc::Log::SpdlogLogger>>>>,const wchar_t(&)[27]>(FacilityIt,FacilityIt,Orc::Log::Level,const wchar_t (&)[27])' being compiled [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          FacilityIt=std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_ptr<Orc::Log::SpdlogLogger>>>>
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log/Log.h(64): message : see reference to function template instantiation 'void Orc::Log::Logger::Error<const wchar_t(&)[27]>(const wchar_t (&)[27])' being compiled (compiling source f
ile C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp(50): message : see reference to function template instantiation 'void Orc::Log::Error<const wchar_t(&)[27]>(const wchar_t (&)[27])' being compiled [C:\Users\forensic
s\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(69,26): error C2664: 'OutputIt fmt::v8::format_to<std::back_insert_iterator<fmt::v8::basic_memory_buffer<wchar_t,500,std::allocator<wchar_t>>>,,0>(OutputIt,fmt::v8::basic_
format_string<char>)': cannot convert argument 2 from 'const wchar_t [26]' to 'fmt::v8::basic_format_string<char>' [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          OutputIt=std::back_insert_iterator<fmt::v8::basic_memory_buffer<wchar_t,500,std::allocator<wchar_t>>>
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(51,5): message : No constructor could take the source type, or constructor overload resolution was ambiguous (compiling source file C:\Users\forensics\source\repos\dfir-or
c\src\OrcLib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\external\vcpkg\installed\x86-windows-static\include\fmt\core.h(3146,17): message : see declaration of 'fmt::v8::format_to' (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcL
ib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(86): message : see reference to function template instantiation 'void Orc::Log::Logger::Log<FacilityIt,std::chrono::system_clock::time_point,const wchar_t(&)[26],>(Facilit
yIt,FacilityIt,const Timepoint &,Orc::Log::Level,Arg0)' being compiled [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          FacilityIt=std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_ptr<Orc::Log::SpdlogLogger>>>>,
          Timepoint=std::chrono::system_clock::time_point,
          Arg0=const wchar_t (&)[26]
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log\Logger.h(188): message : see reference to function template instantiation 'void Orc::Log::Logger::Log<std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_p
tr<Orc::Log::SpdlogLogger>>>>,const wchar_t(&)[26]>(FacilityIt,FacilityIt,Orc::Log::Level,const wchar_t (&)[26])' being compiled [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
      with
      [
          FacilityIt=std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::shared_ptr<Orc::Log::SpdlogLogger>>>>
      ] (compiling source file C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp)
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\Log/Log.h(64): message : see reference to function template instantiation 'void Orc::Log::Logger::Error<const wchar_t(&)[26]>(const wchar_t (&)[26])' being compiled (compiling source f
ile C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp) [C:\Users\forensics\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]
C:\Users\forensics\source\repos\dfir-orc\src\OrcLib\UnitTestHelper.cpp(58): message : see reference to function template instantiation 'void Orc::Log::Error<const wchar_t(&)[26]>(const wchar_t (&)[26])' being compiled [C:\Users\forensic
s\source\repos\dfir-orc\build-x86\src\OrcLib\OrcLib.vcxproj]

The issues are the same for the x64 version.

Any help would be appreciated 🙏

Cheers

fabienfl-orc commented 2 years ago

Hello, thank you for your detailed issue.

1. Missing libwinpthread

It seems to be a recurring issue due to some vcpkg packages, see https://github.com/microsoft/vcpkg/issues/24028

Some dependency pull libpthread but DFIR-Orc should not link with it. If I remember correctly it is boost-multimap which is greedy. I will try to remove that dep in the future.

2. Branch master for vcpkg

It is not possible to build from master branch (or any 'official' commit) because DFIR-Orc does not use vcpkg's new overlay feature. DFIR-Orc forks vcpkg to add custom fixes (see branch 'dfir-orc/2021.12.01'). One can eventually try rebase on vcpkg's master it but it is not always that easy.

cd external/vcpkg
# git clean -fdX  # BEWARE: delete any files which are not comitted (built artefacts...)
git remote add microsoft https://github.com/microsoft/vcpkg.git
git fetch microsoft
git checkout dfir-orc/2021.05.12
git rebase microsoft/master

3. Removing the libwinpthread dep

I am not surprised this seems to work, see 1. Good news.

5. Build issue

Compiler error is trigerred on UnitTestHelper.cpp which is not required. May be switching vcpkg's branch created somme issue ?

I will check myself but I guess it should be working as Azure pipelines run those tests. I the meantime you can try to build DFIR-Orc without tests. Either use a new directory or delete CMakeCache.txt first.

cmake -G "Visual Studio 16 2019" -A Win32 -DORC_BUILD_TEST=OFF ..
fabienfl-orc commented 2 years ago

Removed the dependency to libwinpthread with

diff --git a/ports/fmt/portfile.cmake b/ports/fmt/portfile.cmake
index 39208f6fb..bb26f367c 100644
--- a/ports/fmt/portfile.cmake
+++ b/ports/fmt/portfile.cmake
@@ -42,7 +42,7 @@ endif()
 file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include")

 vcpkg_cmake_config_fixup()
-vcpkg_fixup_pkgconfig()
+#vcpkg_fixup_pkgconfig()  # do not pull pkg-config and msys2 libraries like libwinpthread

 if(VCPKG_TARGET_IS_WINDOWS)
     if(NOT DEFINED VCPKG_BUILD_TYPE OR VCPKG_BUILD_TYPE STREQUAL "debug")
diff --git a/ports/spdlog/portfile.cmake b/ports/spdlog/portfile.cmake
index 3d154cbec..8951b307b 100644
--- a/ports/spdlog/portfile.cmake
+++ b/ports/spdlog/portfile.cmake
@@ -42,7 +42,7 @@ vcpkg_cmake_configure(

 vcpkg_cmake_install()
 vcpkg_cmake_config_fixup(CONFIG_PATH lib/cmake/spdlog)
-vcpkg_fixup_pkgconfig()
+#vcpkg_fixup_pkgconfig()  # do not pull pkg-config and msys2 libraries like libwinpthread
 vcpkg_copy_pdbs()

 # use vcpkg-provided fmt library (see also option SPDLOG_FMT_EXTERNAL above)
NicknameNotTaken commented 2 years ago

Hello,

Thanks a lot for your anwsers. Removing the libwinpthread dependency (in cli11, fmt, and spdlog) and sticking to the fork of vcpkg did it (no need for ORC_BUILD_TEST=OFF). All the cmake operations went smoothly.

Thanks again!