Closed shoe42 closed 2 years ago
Does it help if you build with:
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"
For what it's worth, I confirmed that I get this error on my x86 Mac when I unset SDKROOT
(which I normally have set as shown above).
Looks like that fixes it! I've never had to manually set it before, what part of the build process uses SDKROOT
, out of interest?
A while back, macOS removed /usr/include
. We had a big discussion on that in #365, but the end result was that, as best we can tell, SDKROOT
is the standard way to specify where the development toolkit is located.
As best I can tell, Apple Clang does not need this to be set, because they add custom code on top of vanilla Clang to auto-detect it. In contrast, if you build a vanilla Clang, it will not (by default) detect this; but it will read SDKROOT
if you set it. It may be that Homebrew is setting up a configuration variable in Clang to set this automatically, or else do something the vanilla Clang isn't doing.
But as for the reason why the error occurs with Apple Clang: the auto-detection seems to be finicky. For some reason I don't fully understand, when we use the full path to the C compiler (as we need to do to build LuaJIT), the auto-detection gets shut off. So even though Apple Clang should auto-detect it, you need to set SDKROOT
to make the LuaJIT build work. (Or maybe there is something specific to how the LuaJIT build works, I'm not sure. I haven't really dug too deep into this.) Terra itself should build fine because it's just using the Apple Clang compiler as normal.
But at any rate, you will need SDKROOT
set in order for Terra itself to work because it has the same issues as a vanilla Clang.
Gotcha, interesting quirk to note! Thanks for the detailed info, I can't recall if this is mentioned in the build instructions, but if not it might make a good footnote. Looks like clang might have added auto-detection in https://reviews.llvm.org/D109460?id=371426 through xcrun
, but whether that works or not I can't test. I can't immediately trigger this issue using a test C file containing some of the problematic headers, so it's certainly a strange problem, but I'll take note of this for the future.
By the way, I'd be curious how the test suite looks on M1. You could run either ctest -j4
on a CMake build directory or ../build/bin/terra ./run
from the tests
directory and paste the bottom part of the output (where it shows test pass rate and failing tests).
I presume running both is redundant, but here's the pass rate and failing tests:
ctest -j4
:
82% tests passed, 98 tests failed out of 549
Total Test time (real) = 15.99 sec
The following tests FAILED: 4 - ainline.t (Failed) 21 - arrayt.t (Failed) 22 - arrayt2.t (Failed) 26 - atoi.t (Failed) 27 - atomicrmw.t (Failed) 28 - avxhadd.t (Failed) 30 - benchmark_dgemm.t (Failed) 31 - benchmark_fannkuchredux.t (Failed) 32 - benchmark_nbody.t (Failed) 33 - bf.t (Failed) 36 - blocking.t (Failed) 37 - blocking2-fixed.t (Failed) 38 - blocking2.t (Failed) 39 - blocking3.t (Failed) 41 - bounce.t (Failed) 42 - breaktest.t (Failed) 43 - bug.t (Failed) 44 - bug2.t (Failed) 46 - bug372.t (Failed) 48 - bug372b.t (Failed) 49 - bug372c.t (Failed) 50 - bug372d.t (Failed) 51 - bug4.t (Failed) 56 - canon2.t (Failed) 59 - cconv.t (SEGFAULT) 60 - cconv_array.t (Failed) 61 - cconv_more.t (Failed) 64 - clanginfo.t (Failed) 65 - class.t (Failed) 66 - class2.t (Failed) 67 - class3.t (Failed) 69 - class5.t (Failed) 70 - class6.t (Failed) 72 - clean.t (Failed) 73 - cmpxchg.t (Failed) 77 - compile_time_array.t (Failed) 78 - compile_time_array2.t (Failed) 79 - compile_time_array3.t (Failed) 90 - coverage3.t (Failed) 119 - defer.t (Failed) 120 - deferbreak.t (Failed) 123 - defergoto.t (Failed) 125 - dgemm.t (Failed) 126 - dgemm2.t (Failed) 127 - dgemm3.t (Failed) 129 - diffuse.t (Failed) 130 - dynlib.t (Failed) 144 - exportdynamic.t (Failed) 272 - fakeasm.t (Failed) 299 - gemm.t (Failed) 301 - gettype.t (Failed) 305 - gvarfault.t (Failed) 307 - hello.t (Failed) 308 - hello2.t (Failed) 312 - includec.t (Failed) 334 - linklibrary.t (Failed) 341 - luaapi.t (Failed) 342 - luabridge.t (Failed) 343 - luabridge2.t (Failed) 358 - malloc.t (Failed) 359 - mathlib.t (Failed) 370 - multiterra.t (Failed) 376 - new.t (Failed) 387 - objc.t (Failed) 388 - objc2.t (Failed) 389 - objtest.t (Failed) 392 - option_e.t (Failed) 395 - output.t (Failed) 423 - printfarray.t (Failed) 424 - printfloat.t (Failed) 425 - proxystruct.t (Failed) 427 - pthreads.t (Failed) 435 - quote7.t (Failed) 436 - quote8.t (Failed) 437 - quote9.t (Failed) 440 - quoteselect.t (Failed) 445 - recstruct.t (Failed) 448 - renaming.t (Failed) 451 - scc.t (Failed) 456 - sgemm-old.t (Failed) 457 - sgemm.t (Failed) 458 - sgemm3.t (Failed) 459 - sgemmkernel.t (Failed) 474 - speed.t (Failed) 477 - ssimple.t (Failed) 479 - stattest.t (Failed) 480 - stdio.t (Failed) 481 - stencil.t (Failed) 482 - strerror.t (Failed) 483 - string.t (Failed) 496 - symbolmangling.t (Failed) 518 - testminmax.t (Failed) 519 - testrequire.t (Failed) 520 - teststd.t (Failed) 521 - testvector.t (Failed) 533 - vararg.t (Failed) 534 - varargcstring.t (Failed) 541 - vecobj.t (Failed)
- `../build/bin/terra ./run`:
451 tests passed. 98 tests failed.
Are the tests meant to all pass on any platform?
Hm, this failure rate makes me think you might have forgotten SDKROOT
? Or something similar.
We pass 100% of tests on all x86 platforms: macOS, Linux, FreeBSD, and Windows. We also pass 100% of tests on PPC64le Linux.
On ARM64 Linux, the pass rate is around 96% (https://github.com/terralang/terra/issues/565#issuecomment-1153306721). Even in emulation I get 95% pass rate (https://github.com/terralang/terra/issues/565#issuecomment-1153471338).
I'm hoping that there's a simple fix (like SDKROOT
) that gets you up to a similar pass rate.
SDKROOT
doesn't seem to fix it - not sure what's going on here. I'll see if I can dig a bit deeper, but at least some seem to be due to missing headers (with SDKROOT
set):
stack traceback:
[C]: in function 'registercfile'
.../terra/src/terralib.lua:3529: in function 'includecstring'
./lib/objc.t:10: in main chunk
[C]: in function 'require'
testrequire.t:9: in main chunk
<buffer>:1:10: fatal error: 'stdio.h' file not found
#include "stdio.h"
^~~~~~~~~
compilation of included c code failed
I can't explain why SDKROOT
wouldn't be working, but that failure mode looks exactly like what I'd expect if SDKROOT
weren't set.
You might be able to work around with:
export INCLUDE_PATH="$(xcrun --sdk macosx --show-sdk-path)/usr/include"
That's a different way of specifying the path and shouldn't be necessary if SDKROOT
is set. But it might work if SDKROOT
isn't.
You built with an LLVM from Homebrew, right? It may also be worth trying a vanilla LLVM (there's a build recipe here: https://github.com/terralang/terra#instructions-for-building-llvm-from-source).
That seems to fix things up (manually setting INCLUDE_PATH
) - just having SDKROOT
set doesn't seem to let it find the headers. I'm using the prebuilt LLVM 13 from Homebrew (llvm@13
). Pass rate is now much better:
96% tests passed, 21 tests failed out of 549
Total Test time (real) = 4.98 sec
The following tests FAILED:
27 - atomicrmw.t (Subprocess aborted)
46 - bug372.t (SEGFAULT)
48 - bug372b.t (SEGFAULT)
49 - bug372c.t (SEGFAULT)
50 - bug372d.t (SEGFAULT)
56 - canon2.t (Failed)
59 - cconv.t (SEGFAULT)
60 - cconv_array.t (Failed)
61 - cconv_more.t (Failed)
65 - class.t (Failed)
67 - class3.t (SEGFAULT)
70 - class6.t (Failed)
90 - coverage3.t (Failed)
272 - fakeasm.t (Failed)
301 - gettype.t (Failed)
342 - luabridge.t (Failed)
343 - luabridge2.t (Failed)
423 - printfarray.t (SEGFAULT)
477 - ssimple.t (Failed)
533 - vararg.t (Subprocess aborted)
541 - vecobj.t (Failed)
Thanks. It's good to know we at least have a path to getting it mostly working.
I'd be surprised if SDKROOT
parsing is really dependent on x86, but I suppose it's possible. (If so, probably an oversight.) My only other idea is to try a vanilla source build since Homebrew does sometimes add patches to their packages. (And again, instructions at https://github.com/terralang/terra#instructions-for-building-llvm-from-source .)
@shoe42 by the way, if you pull from master, you should (hopefully) see a higher test pass rate now that #593 has merged.
@elliottslaughter apologies for the delay - from the current master branch (23559ea), using self-built LLVM 13 as per the instructions in the Terra docs, it still seems to fail to find the headers. Setting SDKROOT
works as expected. Pass rate seems to be better:
$ ctest -j4
.....
98% tests passed, 10 tests failed out of 549
Total Test time (real) = 15.44 sec
The following tests FAILED:
27 - atomicrmw.t (Subprocess aborted)
46 - bug372.t (SEGFAULT)
48 - bug372b.t (SEGFAULT)
49 - bug372c.t (SEGFAULT)
60 - cconv_array.t (Failed)
65 - class.t (SEGFAULT)
67 - class3.t (SEGFAULT)
272 - fakeasm.t (Failed)
423 - printfarray.t (SEGFAULT)
533 - vararg.t (Subprocess aborted)
Thanks for testing.
I should have mentioned #597. On Linux, this means that AArch64 only works with LLVM 11. I suspect this would help you fix most or all of these outstanding failures.
On an M1 mac, running macOS 12.5 and with clitools version 13.0.0.0.1.1627064638, if you try and use the stock clitools clang, the build process fails to find some headers - see below log. My current workaround is to use clang from Homebrew's
llvm@13
.Failing build:
Running
make
a 2nd time to try and just get the errors outputs the following:Some other info: