run ci on macos-latest

[jnikula]

Responding to https://github.com/jnikula/hawkmoth/pull/248#issuecomment-2347139425 here:

Looks like we need to specify the C++ standard to use for a lot of tests. The default is different on macos. I just added -std=c++17 to the .yaml files of a bunch of tests. Is that okay, or should we add a default in testenv.py that could be overridden in .yaml?

Other than that, assuming a default may be wise (perhaps the difference is in the clang version each system uses by default, in which case it doesn't hurt to be explicit). Adding it in every test case is not so good I think as the test developer may not immediately know he needs to.

Agreed. Maybe we should specify the C standard too.

Still having include issues, and it's driving me nuts. Xcode libclang can't find C++ headers, but homebrew version can. Xcode libclang can find C headers, but homebrew version can't. If one or the other worked, I'd just call it a day.

Alas I know nothing of macos, but that's weird indeed. How are you testing this? The CI itself or do you have a system on your end? I.e. can I see the output?

That's the thing, I don't have a system. Completely depending on the github workflow for debugging, which is not great.

[BrunoMSantos]

This one's pretty odd indeed, but I'm not sure I have much to offer in help.

I found this, where the 'correct answer' may be a clue to something: https://stackoverflow.com/questions/38960314/getting-fatal-error-type-traits-file-not-found-include-type-traits-while/38966585#38966585

Otherwise, have you tried printing out the actual search paths being used and check the files are there? Just create a dummy test sequence that does that.

Something like clang -E -v -### -x c /dev/null and clang -E -v -### -x c++ /dev/null should work.

[jnikula]

Xcode C

Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" "-cc1" "-triple" "arm64-apple-macosx14.0.0" "-Wundef-prefix=TARGET_OS_" "-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage" "-Werror=implicit-function-declaration" "-fsyntax-only" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "simple-macro.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=non-leaf" "-fno-strict-return" "-fallow-editor-placeholders" "-ffp-contract=on" "-fno-rounding-math" "-funwind-tables=1" "-fobjc-msgsend-selector-stubs" "-fcompatibility-qualified-id-block-type-checking" "-fvisibility-inlines-hidden-static-local-var" "-target-cpu" "apple-m1" "-target-feature" "+v8.5a" "-target-feature" "+crc" "-target-feature" "+lse" "-target-feature" "+rdm" "-target-feature" "+crypto" "-target-feature" "+dotprod" "-target-feature" "+fp-armv8" "-target-feature" "+neon" "-target-feature" "+fp16fml" "-target-feature" "+ras" "-target-feature" "+rcpc" "-target-feature" "+zcm" "-target-feature" "+zcz" "-target-feature" "+fullfp16" "-target-feature" "+sm4" "-target-feature" "+sha3" "-target-feature" "+sha2" "-target-feature" "+aes" "-target-abi" "darwinpcs" "-debugger-tuning=lldb" "-target-linker-version" "1053.12" "-v" "-fcoverage-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-resource-dir" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include" "-internal-externc-isystem" "/usr/include" "-internal-externc-isystem" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include" "-Wno-reorder-init-list" "-Wno-implicit-int-float-conversion" "-Wno-c99-designator" "-Wno-final-dtor-non-final-class" "-Wno-extra-semi-stmt" "-Wno-misleading-indentation" "-Wno-quoted-include-in-framework-header" "-Wno-implicit-fallthrough" "-Wno-enum-enum-conversion" "-Wno-enum-float-conversion" "-Wno-elaborated-enum-base" "-Wno-reserved-identifier" "-Wno-gnu-folding-constant" "-fdebug-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-ferror-limit" "19" "-stack-protector" "1" "-fstack-check" "-mdarwin-stkchk-strong-link" "-fblocks" "-fencode-extended-block-signature" "-fregister-global-dtors-with-atexit" "-fgnuc-version=4.2.1" "-fmax-type-align=16" "-fcommon" "-fno-spell-checking" "-detailed-preprocessing-record" "-clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation" "-fno-odr-hash-protocols" "-clang-vendor-feature=+enableAggressiveVLAFolding" "-clang-vendor-feature=+revert09abecef7bbf" "-clang-vendor-feature=+thisNoAlignAttr" "-clang-vendor-feature=+thisNoNullAttr" "-mllvm" "-disable-aligned-alloc-awareness=1" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-x" "c" "/Users/runner/work/github-actions-playground/github-actions-playground/test/c/simple-macro.c"

[jnikula]

Xcode C++

Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
ignoring nonexistent directory "/usr/include/c++/v1"
ignoring nonexistent directory "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1"
 "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" "-cc1" "-triple" "arm64-apple-macosx14.0.0" "-Wundef-prefix=TARGET_OS_" "-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage" "-Werror=implicit-function-declaration" "-fsyntax-only" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "simple-macro.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=non-leaf" "-fno-strict-return" "-fallow-editor-placeholders" "-ffp-contract=on" "-fno-rounding-math" "-funwind-tables=1" "-fobjc-msgsend-selector-stubs" "-fcompatibility-qualified-id-block-type-checking" "-fvisibility-inlines-hidden-static-local-var" "-target-cpu" "apple-m1" "-target-feature" "+v8.5a" "-target-feature" "+crc" "-target-feature" "+lse" "-target-feature" "+rdm" "-target-feature" "+crypto" "-target-feature" "+dotprod" "-target-feature" "+fp-armv8" "-target-feature" "+neon" "-target-feature" "+fp16fml" "-target-feature" "+ras" "-target-feature" "+rcpc" "-target-feature" "+zcm" "-target-feature" "+zcz" "-target-feature" "+fullfp16" "-target-feature" "+sm4" "-target-feature" "+sha3" "-target-feature" "+sha2" "-target-feature" "+aes" "-target-abi" "darwinpcs" "-debugger-tuning=lldb" "-target-linker-version" "1053.12" "-v" "-fcoverage-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-resource-dir" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include" "-internal-externc-isystem" "/usr/include" "-internal-externc-isystem" "/Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include" "-Wno-reorder-init-list" "-Wno-implicit-int-float-conversion" "-Wno-c99-designator" "-Wno-final-dtor-non-final-class" "-Wno-extra-semi-stmt" "-Wno-misleading-indentation" "-Wno-quoted-include-in-framework-header" "-Wno-implicit-fallthrough" "-Wno-enum-enum-conversion" "-Wno-enum-float-conversion" "-Wno-elaborated-enum-base" "-Wno-reserved-identifier" "-Wno-gnu-folding-constant" "-fdeprecated-macro" "-fdebug-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-ferror-limit" "19" "-stack-protector" "1" "-fstack-check" "-mdarwin-stkchk-strong-link" "-fblocks" "-fencode-extended-block-signature" "-fregister-global-dtors-with-atexit" "-fgnuc-version=4.2.1" "-fno-cxx-modules" "-fcxx-exceptions" "-fexceptions" "-fmax-type-align=16" "-fcommon" "-fno-spell-checking" "-detailed-preprocessing-record" "-clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation" "-fno-odr-hash-protocols" "-clang-vendor-feature=+enableAggressiveVLAFolding" "-clang-vendor-feature=+revert09abecef7bbf" "-clang-vendor-feature=+thisNoAlignAttr" "-clang-vendor-feature=+thisNoNullAttr" "-mllvm" "-disable-aligned-alloc-awareness=1" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-x" "c++" "/Users/runner/work/github-actions-playground/github-actions-playground/test/c/simple-macro.c"

[jnikula]

Homebrew C

Homebrew clang version 15.0.7
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: 
 "clang" "-cc1" "-triple" "arm64-apple-macosx14.0.0" "-Wundef-prefix=TARGET_OS_" "-Werror=undef-prefix" "-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage" "-fsyntax-only" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "simple-macro.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=non-leaf" "-fallow-editor-placeholders" "-ffp-contract=on" "-fno-rounding-math" "-funwind-tables=2" "-fcompatibility-qualified-id-block-type-checking" "-fvisibility-inlines-hidden-static-local-var" "-target-cpu" "apple-m1" "-target-feature" "+v8.5a" "-target-feature" "+crc" "-target-feature" "+lse" "-target-feature" "+rdm" "-target-feature" "+crypto" "-target-feature" "+dotprod" "-target-feature" "+fp-armv8" "-target-feature" "+neon" "-target-feature" "+fp16fml" "-target-feature" "+ras" "-target-feature" "+rcpc" "-target-feature" "+zcm" "-target-feature" "+zcz" "-target-feature" "+fullfp16" "-target-feature" "+sm4" "-target-feature" "+sha3" "-target-feature" "+sha2" "-target-feature" "+aes" "-target-abi" "darwinpcs" "-fallow-half-arguments-and-returns" "-mllvm" "-treat-scalable-fixed-error-as-warning" "-debugger-tuning=lldb" "-target-linker-version" "1022.1" "-v" "-fcoverage-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-resource-dir" "lib/clang/15.0.7" "-isysroot" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk" "-internal-isystem" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/local/include" "-internal-isystem" "lib/clang/15.0.7/include" "-internal-externc-isystem" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include" "-fdebug-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-ferror-limit" "19" "-stack-protector" "1" "-fblocks" "-fencode-extended-block-signature" "-fregister-global-dtors-with-atexit" "-fgnuc-version=4.2.1" "-fmax-type-align=16" "-fno-spell-checking" "-detailed-preprocessing-record" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-x" "c" "/Users/runner/work/github-actions-playground/github-actions-playground/test/c/simple-macro.c"

[jnikula]

Homebrew C++

Homebrew clang version 15.0.7
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: 
ignoring nonexistent directory "../include/c++/v1"
 "clang" "-cc1" "-triple" "arm64-apple-macosx14.0.0" "-Wundef-prefix=TARGET_OS_" "-Werror=undef-prefix" "-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage" "-fsyntax-only" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "simple-macro.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=non-leaf" "-fallow-editor-placeholders" "-ffp-contract=on" "-fno-rounding-math" "-funwind-tables=2" "-fcompatibility-qualified-id-block-type-checking" "-fvisibility-inlines-hidden-static-local-var" "-target-cpu" "apple-m1" "-target-feature" "+v8.5a" "-target-feature" "+crc" "-target-feature" "+lse" "-target-feature" "+rdm" "-target-feature" "+crypto" "-target-feature" "+dotprod" "-target-feature" "+fp-armv8" "-target-feature" "+neon" "-target-feature" "+fp16fml" "-target-feature" "+ras" "-target-feature" "+rcpc" "-target-feature" "+zcm" "-target-feature" "+zcz" "-target-feature" "+fullfp16" "-target-feature" "+sm4" "-target-feature" "+sha3" "-target-feature" "+sha2" "-target-feature" "+aes" "-target-abi" "darwinpcs" "-fallow-half-arguments-and-returns" "-mllvm" "-treat-scalable-fixed-error-as-warning" "-debugger-tuning=lldb" "-target-linker-version" "1022.1" "-v" "-fcoverage-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-resource-dir" "lib/clang/15.0.7" "-isysroot" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk" "-stdlib=libc++" "-internal-isystem" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/c++/v1" "-internal-isystem" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/local/include" "-internal-isystem" "lib/clang/15.0.7/include" "-internal-externc-isystem" "/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include" "-fdeprecated-macro" "-fdebug-compilation-dir=/Users/runner/work/github-actions-playground/github-actions-playground" "-ferror-limit" "19" "-stack-protector" "1" "-fblocks" "-fencode-extended-block-signature" "-fregister-global-dtors-with-atexit" "-fgnuc-version=4.2.1" "-fcxx-exceptions" "-fexceptions" "-fmax-type-align=16" "-fno-spell-checking" "-detailed-preprocessing-record" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-x" "c++" "/Users/runner/work/github-actions-playground/github-actions-playground/test/c/simple-macro.c"

[jnikula]

I added this to a test yaml file, and grabbed the outputs above:

  options:
    clang:
    - -v
    - -###

[jnikula]

This seems to be a cool way to debug it: https://github.com/namespacelabs/breakpoint

[BrunoMSantos]

Well, it already complains about some missing directories... Now why I don't have a clue. Is there a way we can inspect the filesystem in the CI image?

Sorry I haven't been more proactive, but haven't had the time to do more than dropping a few random comments.

[jnikula]

Just pushed a rebased version, no changes in the results, and none expected.

Well, it already complains about some missing directories... Now why I don't have a clue. Is there a way we can inspect the filesystem in the CI image?

I've toyed with the breakpoint action I mentioned above. It keeps the workflow running and lets you ssh into the runner. You can actually run the tests interactively, and reproduce like that. Haven't gotten far though.

Some of the missing directories are due to Apple removing /usr/include etc. to support multiple SDK installations.

We used to run clang (via hawkmoth.util.compiler.get_include_args()) to figure out the include paths, and then pass them on to libclang, but I've completely changed my mind about that. Which is why I removed it from tests. It's so easy to get it wrong. (I was hitting the warnings added in https://reviews.llvm.org/D131441 for instance.) libclang should figure it all out itself. And in some cases in the Docker containers the problem was that the C or C++ headers weren't pulled in automatically, and needed to be explicitly installed.

Sorry I haven't been more proactive, but haven't had the time to do more than dropping a few random comments.

Don't worry about it, and I do appreciate you having even a glance at the pull requests.