terralang / terra

Terra is a low-level system programming language that is embedded in and meta-programmed by the Lua programming language.
terralang.org
Other
2.72k stars 201 forks source link

Get system include paths from clang instead of hardcoding them #412

Closed ErikMcClure closed 5 years ago

ErikMcClure commented 5 years ago

This uses a hack from Dragon FFI to extract clang's own system header files, avoiding the necessity of trying to find them ourselves. It then deletes genclangpaths.lua and removes clangpaths.h from both cmake and makefile builds, fixing the hardcoded paths problem.

This pull request also makes some minor changes to windows-specific hacks that were put in place, removing them for newer versions that no longer require these workarounds to promote code homogeneity. In particular, we no longer use the Visual Studio version to set fms-compatibility-version, because this is actually the version of cl.exe, not the SDK, or Visual Studio, or anything else. It may be preferable to try and let clang figure this value out on it's own, because the logic does exist, but is inaccessible. For now, a value of 1800 is fine for the purposes of parsing most windows headers, and is better than simply using "whatever version of visual studio we happened to compile with".

Incidentally, this fixed my own problems with compiling Terra on Ubuntu 19, so this pull request may have accidentally fixed #393, but someone will need to verify this. I've tested this pull request on both Windows 10 and Ubuntu 19, but we will need to see if I broke any of the test environments, since this is a nontrivial change to the build process.

elliottslaughter commented 5 years ago

This is awesome! Thanks for doing this.

There is one glitch I noticed where the CUDA tests seem to be broken by this. I can build, but then CUDA somehow ends up not being put on the system include path. Setting INCLUDE_PATH manually avoids the problem, but really seems like an unfortunate compromise given what we're trying to do here.

$ ../terra cudaoo.t 
<buffer>:1:10: fatal error: 'cuda_runtime.h' file not found
#include "cuda_runtime.h"
         ^~~~~~~~~~~~~~~~
compilation of included c code failed

stack traceback:
    [C]: in function 'registercfile'
    src/terralib.lua:3412: in function 'includec'
    cudaoo.t:9: in main chunk

$ INCLUDE_PATH=/usr/local/cuda-9.2/targets/x86_64-linux/include ../terra cudaoo.t

I assume we're just missing some sort of Clang configuration to pick up all the system include directories. Do you have any interest/availability to look into this?

ErikMcClure commented 5 years ago

What you are expecting is not what Terra is doing, and I don't think Terra actually ever did what you described except by accident. The CUDA header logic was preserved in terralib.lua, but it tries to use $CUDA_HOME$ to find CUDA on Linux.

if ffi.os == "Windows" then
  terra.cudahome = os.getenv("CUDA_PATH")
else
  terra.cudahome = os.getenv("CUDA_HOME") or "/usr/local/cuda"
end

If you want to change how Terra finds CUDA and try to convince clang itself to look up the include directory, this is hard to test because none of the current tests you have set up actually do things this way. I've pushed a change where I call AddCudaIncludeArgs, but I have no idea if this will work and have no test to verify it against.