apple m1 support

[bigdogs]

Hi, Is there a possible to support apple M1 natively?

[lilianmoraru]

Qt support for Apple M1 is currently planned for Qt 6.2: https://bugreports.qt.io/browse/QTBUG-85279

[variar]

I don't have any Apple computers. All Mac-specific development is done using Virtual Box VM. If someone can point me to a guide about M1 installation inside VM, I'll see what can be done.

[earslap]

Hi @variar thank you (and the contributors) for the great software. Just wanted to let you know that you don't need an M1 machine (or VM running arm version of macOS) to build universal binaries (Intel + M1 fat binaries) for Mac. I'm sure there will be other issues regarding Qt and your specific build setup that I don't yet know about, but wanted to drop this here in case it helps somewhat in the future:

https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

You'll only be able to test the Intel slice of the application of course. But M1 users can help with the rest I assume.

[henrify]

@variar I've been a long time user of glogg and just found your project, great work.

QT 6.2 was released in September.

Could you update klogg to build/work with QT 6 so that users can build themselves for M1?

Currently... cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DQt5_DIR=/opt/homebrew/Cellar/qt/6.2.3_1/lib/cmake/Qt6 .. Gives... By not providing "FindQt5.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "Qt5", but CMake did not find one.

[variar]

There is an open PR that allows building with Qt6, but I still cat't find time to test it.

[variar]

@henrify I've merge Qt6 support to master. On linux I configure it like this

cmake -G Ninja -DCMAKE_PREFIX_PATH=/home/omni/projects/sdk/Qt/6.2.3/gcc_64/  -DCMAKE_BUILD_TYPE=RelWithDebInfo -DKLOGG_USE_HYPERSCAN=False ..

Unfortunately, hyperscan regex engine doesn't support M1, so it has to be disabled. There is a fork vectorscan, but I don't know if it is stable enough.

Please try it out and tell me if something is broken.

[henrify]

@variar Thanks for the fast reply and the merge.

I gave it a try, but with Big Sur (11.6) and XCode 13.2.1 I got an error about missing headers for pthread.

I've verified that command line tools are installed and I got pthread.h in e.g. /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/pthread.h -> pthread/pthread.h but seems that Klogg's build doesn't find it.

The problem doesn't seem to be specific to Klogg and it is discussed in: https://developer.apple.com/forums/thread/666700 but I'm not sure how to apply the solution (=configure Klogg's build to set the path).

The exact error message is as follows:

air /Volumes/Files/Projects/_ByOthers/klogg/build_root $ cmake -G Ninja -DCMAKE_PREFIX_PATH=/home/omni/projects/sdk/Qt/6.2.3/gcc_64/  -DCMAKE_BUILD_TYPE=RelWithDebInfo -DKLOGG_USE_HYPERSCAN=False ..
-- The C compiler identification is AppleClang 13.0.0.13000029
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Version override 
Build 
Project version is 22.3.0
Project version tweak is 0
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) 
Flags set for 'AppleClang' compiler:-ffast-math;-stdlib=libc++
Defines set for 'AppleClang' compiler:

CMAKE_C_FLAGS:  -mmmx -msse -msse2 -msse3 -mssse3 -mpopcnt -march=native -mtune=generic
CMAKE_CXX_FLAGS:  -fno-sized-deallocation -fno-omit-frame-pointer -mmmx -msse -msse2 -msse3 -mssse3 -mpopcnt -march=native -mtune=generic
CMAKE_EXE_LINKER_FLAGS: 
CMAKE_MODULE_LINKER_FLAGS: 
CMAKE_SHARED_LINKER_FLAGS: 
CMAKE_STATIC_LINKER_FLAGS: 
CMAKE_C_FLAGS_RELWITHDEBINFO: -O2 -g -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO: -O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO: 
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO: 
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO: 
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO: 

-- Looking for pthread.h
-- Looking for pthread.h - not found
CMake Error at /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindThreads.cmake:238 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /opt/homebrew/lib/cmake/Qt6/Qt6Dependencies.cmake:48 (find_dependency)
  /opt/homebrew/lib/cmake/Qt6/Qt6Config.cmake:140 (include)
  CMakeLists.txt:151 (find_package)

-- Configuring incomplete, errors occurred!
See also "/Volumes/Files/Projects/_ByOthers/klogg/build_root/CMakeFiles/CMakeOutput.log".
See also "/Volumes/Files/Projects/_ByOthers/klogg/build_root/CMakeFiles/CMakeError.log".

[variar]

From discussion on that link it looks like setting LDFLAGS environment variable in the terminal could help. Try something like this:

export LDFLAGS=-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=/opt/homebrew/Cellar/qt/6.2.3_1 ..

[henrify]

Thanks, but doesn't work, I did try couple of the suggestion from that thread (that's why I said I don't know how to "apply" the solution to Klogg build).

Which macOS version you use in CI?

[variar]

CI is using GitHub actions. Currently it is macOS Big Sur 11.

Looking at the error one more time, and I don't see how adding LDFLAGS could help finding pthread.h. Maybe adding LDFLAGS and adding this to klogg's root CMakeLists.txt could help

IF(APPLE)
    set(CMAKE_THREAD_LIBS_INIT "-lpthread")
    set(CMAKE_HAVE_THREADS_LIBRARY 1)
    set(CMAKE_USE_WIN32_THREADS_INIT 0)
    set(CMAKE_USE_PTHREADS_INIT 1)
    set(THREADS_PREFER_PTHREAD_FLAG ON)
ENDIF()

[henrify]

Thanks, that fixes the build file config/generation step, but the actual build step fails immediately.

Same error repeats for about half of the 3rdparty libs:

air /Volumes/Files/Projects/_ByOthers/klogg/build_root $ cmake --build .
[3/368] Building C object 3rdparty/roaring/CMakeFiles/roaring.dir/src/roaring.c.o
FAILED: 3rdparty/roaring/CMakeFiles/roaring.dir/src/roaring.c.o 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc  -I/Volumes/Files/Projects/_ByOthers/klogg/build_root/generated -isystem /Volumes/Files/Projects/_ByOthers/klogg/3rdparty/roaring/include -mmmx -msse -msse2 -msse3 -mssse3 -mpopcnt -march=native -mtune=generic -O2 -g -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=10.14 -MD -MT 3rdparty/roaring/CMakeFiles/roaring.dir/src/roaring.c.o -MF 3rdparty/roaring/CMakeFiles/roaring.dir/src/roaring.c.o.d -o 3rdparty/roaring/CMakeFiles/roaring.dir/src/roaring.c.o -c /Volumes/Files/Projects/_ByOthers/klogg/3rdparty/roaring/src/roaring.c
clang: error: the clang compiler does not support '-march=native'
[4/368] Building CXX object 3rdparty/backward-cpp/CMakeFiles/backward_object.dir/backward.cpp.o

[variar]

Thanks for keep trying this. I don't have mac hardware, so your help in testing is very appreciated. Klogg by default is build with tuning for the build host cpu. Looks like it will not work for m1 builds. In root CMakeLists.txt there is this block:

if (KLOGG_GENERIC_CPU)
        ucm_add_flags(C CXX "-march=x86-64 -mtune=generic")
else()
        ucm_add_flags(C CXX "-march=native -mtune=generic")
endif()

Try commenting out these lines, that should remove -march from build flags.

[henrify]

Thanks again, solved that error. Then next error...

...Was caused by too low macOS version in CMAKE_OSX_DEPLOYMENT_TARGET.

Changing version to 11.3, solved that error and the build started. Then next error...

...Every source file gave warnings about CPU features (mmx etc), until suddenly klogg_version.cpp gave the same whine, but as error, which stopped the build.

[213/368] Building C object 3rdparty/karchive/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/simple/simple_coder.c.o
clang: warning: argument unused during compilation: '-mmmx' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-msse' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-msse2' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-msse3' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-mssse3' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-mpopcnt' [-Wunused-command-line-argument]
[214/368] Building CXX object src/klogg_version/CMakeFiles/klogg_version.dir/src/klogg_version.cpp.o
FAILED: src/klogg_version/CMakeFiles/klogg_version.dir/src/klogg_version.cpp.o 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DQT_CORE_LIB -DQT_NO_DEBUG -I/Volumes/Files/Projects/_ByOthers/klogg/build_root/generated -I/Volumes/Files/Projects/_ByOthers/klogg/src/klogg_version/include -iframework /opt/homebrew/lib -isystem /opt/homebrew/lib/QtCore.framework/Headers -isystem /opt/homebrew/share/qt/mkspecs/macx-clang -isystem /opt/homebrew/include -fno-sized-deallocation -fno-omit-frame-pointer -mmmx -msse -msse2 -msse3 -mssse3 -mpopcnt -O2 -g -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.3 -ffast-math -stdlib=libc++ -Wall -Wextra -Wshadow -Wnon-virtual-dtor -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wdouble-promotion -Wformat=2 -Werror -std=gnu++17 -MD -MT src/klogg_version/CMakeFiles/klogg_version.dir/src/klogg_version.cpp.o -MF src/klogg_version/CMakeFiles/klogg_version.dir/src/klogg_version.cpp.o.d -o src/klogg_version/CMakeFiles/klogg_version.dir/src/klogg_version.cpp.o -c /Volumes/Files/Projects/_ByOthers/klogg/src/klogg_version/src/klogg_version.cpp
clang: error: argument unused during compilation: '-mmmx' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-msse' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-msse2' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-msse3' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-mssse3' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-mpopcnt' [-Werror,-Wunused-command-line-argument]

Commenting out line ucm_add_flags(C CXX "-mmmx -msse -msse2 -msse3 -mssse3 -mpopcnt"), solved that error and the build progressed further. Then next error...

...Is again related to CPU features, or my commenting out of the CPU flags:

[252/368] Building CXX object src/crash_handler/CMakeFiles/klogg_crash_handler.dir/src/cpu_info.cpp.o
FAILED: src/crash_handler/CMakeFiles/klogg_crash_handler.dir/src/cpu_info.cpp.o 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DQT_CONCURRENT_LIB -DQT_CORE_LIB -DQT_GUI_LIB -DQT_MESSAGELOGCONTEXT -DQT_NETWORK_LIB -DQT_NO_DEBUG -DQT_WIDGETS_LIB -I/Volumes/Files/Projects/_ByOthers/klogg/build_root/src/crash_handler/klogg_crash_handler_autogen/include -I/Volumes/Files/Projects/_ByOthers/klogg/build_root/generated -I/Volumes/Files/Projects/_ByOthers/klogg/src/crash_handler/include -I/Volumes/Files/Projects/_ByOthers/klogg/src/klogg_version/include -I/Volumes/Files/Projects/_ByOthers/klogg/src/utils/include -I/Volumes/Files/Projects/_ByOthers/klogg/src/logging/include -iframework /opt/homebrew/lib -isystem /opt/homebrew/lib/QtCore.framework/Headers -isystem /opt/homebrew/share/qt/mkspecs/macx-clang -isystem /opt/homebrew/include -isystem /opt/homebrew/lib/QtGui.framework/Headers -isystem /opt/homebrew/lib/QtWidgets.framework/Headers -isystem /opt/homebrew/lib/QtNetwork.framework/Headers -isystem /opt/homebrew/lib/QtConcurrent.framework/Headers -isystem /Volumes/Files/Projects/_ByOthers/klogg/3rdparty/whereami/include -fno-sized-deallocation -fno-omit-frame-pointer -O2 -g -DNDEBUG -flto=thin -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.3 -ffast-math -stdlib=libc++ -Wall -Wextra -Wshadow -Wnon-virtual-dtor -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wdouble-promotion -Wformat=2 -Werror -std=gnu++17 -MD -MT src/crash_handler/CMakeFiles/klogg_crash_handler.dir/src/cpu_info.cpp.o -MF src/crash_handler/CMakeFiles/klogg_crash_handler.dir/src/cpu_info.cpp.o.d -o src/crash_handler/CMakeFiles/klogg_crash_handler.dir/src/cpu_info.cpp.o -c /Volumes/Files/Projects/_ByOthers/klogg/src/crash_handler/src/cpu_info.cpp
/Volumes/Files/Projects/_ByOthers/klogg/src/crash_handler/src/cpu_info.cpp:92:10: error: use of undeclared identifier '__builtin_cpu_supports'
    if ( __builtin_cpu_supports( "avx512f" ) ) {
         ^

[variar]

Yea, sorry about that :(. Klogg has many x86-64 specific things as it is the only supported platform. And I don't know if roaring-bitmap project is supported for M1. Could be hard to replace it.

[EETagent]

It is indeed possible to compile for arm64 with Hyperscan disabled (I haven't tried replacing it with vectorscan yet, but it should work). The program seems to work well, but I'm no klogg poweruser, discovered this program quite recently.

[EETagent]

The hyperscan support is quite simple, replacing the git repository in cpm.

You can try the program (Built for macOS 13.4)

(Built with neon flags => -march=armv8-a+simd -mcpu=native -mfpu=neon -mfloat-abi=hard -Qunused-arguments)

klogg.zip

[leigeber]

@EETagent Loads up and functions as expected for me. Excited to have a path forward with this. Nice work 🏆

[variar]

This is great news! The next step would be to setup a CI build to produce the packages for M1.

[variar]

I've added a cmake option to buld with vectorscan. Adding to CI should be possible after github/roadmap#528

[rterbush]

I can confirm that the binary shared by @EETagnet works perfectly. Looking forward to support for arm64.