Make it possible to recover include directories from dependency object in portable way

[andrzejc]

The rationale is as follows: I work on a project which uses NVCC to compile .cu files, for which nvcc is used as a generator(); the .cu files are compiled into host & device code, and their host code uses Boost headers. Boost is added to the project as a dependency('boost'). In order for nvcc to compile the .cu files without error I need to provide path to Boost include directory as a command line argument to the generator invocation, but Boost is abstracted away as a dependency which makes it only usable with the regular meson executable/library targets. My current workaround (which is very hackish, but works) is to rely on the knowledge which boost headers define functions returning __FILE__ macro:

  boost_dep = dependency('boost')
  boost_include_dir = cpp.run('''
    #include <boost/log/trivial.hpp>
    #include <iostream>
    int main() {
      std::cout << boost::log::trivial::logger::registration_file();
      return 0;
    }
   ''', dependencies: boost_dep).stdout().substring(0, -22)

The perfect solution would be to have built-in support for CUDA compilation working with dependency() objects or their partial dependencies to obtain the required include directories, but the only related thing I can find in the docs is reference to "Unstable CUDA module" which isn't what I need. The less optimal, but IMHO still elegant solution would be to allow recovering include directories from dependency tree as a flattened list of strings, which would be allow to pass them to generators like NVCC or SWIG. This should be possible regardless of the type of the wrapped dependency. Please let me know if there is any other way of achieving this apart from the described hack (which works only for few select dependencies that actually publicly use __FILE__ in their headers).

[dcbaker]

Can you not use the builtin cuda compilation support instead of using a generator?

[andrzejc]

Thank you, I wasn't aware meson has builtin CUDA support. As mentioned, when I search for meson+CUDA, what I find is https://mesonbuild.com/Cuda-module.html - maybe the existence of builtin CUDA support should be somehow emhasized in the documentation? I definitely will try using the builtin support and I'll be happy to close this issue if that works. Nevertheless, obtaining the list of include directories from a dependency is an useful feature which will greatly enhance interoperability with other generators (I already mentioned SWIG, another what immediately comes to my mind is building of Python extension modules or translating CUDA code to AMD/HIP). I wouldn't categorize this request/proposal as CUDA-specific.

[dcbaker]

We do have a helper in the python module for building python extensions, so you shouldn't need to extract the versions there. The biggest problem with extracting the include_directories objects is we don't know what form to pass them in. Does your compiler use /idir, or ['/i', 'dir'], or '-idir', or --include=dir, or something else?

We probably should have a helper for SWIG, but I don't know if we have anyone using SWIG in meson yet (I'm always surprised to find projects using meson that I had no idea).

[andrzejc]

I don't think the compiler's argument syntax should matter here, because it should be the user's responsibility to pass the arguments to the generator in a form that it accepts. So I'd simply return a list of paths without any decoration, and leave the formatting to the person integrating generator into the build script.

[jpakkane]

One of the main design points of Meson has been to solve peoples' problems rather than try to give them tools to do it themselves. The reason for this is that then everyone has to solve their own problems, usually by copypaste cargoculting code from one project to another. For things like comping Cuda or Python extensions the much better approach is that we solve it once, put in in Meson with unit tests and all that stuff. This way every project can use that rather than wasting their time massaging compiler command lines by hand.

[andrzejc]

I understand your point and I agree that this would be the best if people didn't have to reinvent the wheels all the time, but the harsh reality is that you'll never keep up with the pace of introduction of new development tools and technologies (AMD HIP compiler is a good example), so IMHO it's better to give the users the tools to do sth by themselves than leave them empty handed. I'm very eager to convert my project to stop using the generator() approach and use built-in CUDA support, but as of now it doesn't work, so my choices are: continue using generator(), wait indefinite time until the possible bug in meson CUDA support is fixed, or change the buildsystem to one which doesn't stand in my way and allows me to do my work. As much as I appreciate your commitment to creating reusable, beutiful designs, any solution that works is better than perfectly designed solution which doesn't work.

[denizzzka]

@jpakkane

For things like comping Cuda or Python extensions the much better approach is that we solve it once, put in in Meson with unit tests and all that stuff. This way every project can use that rather than wasting their time massaging compiler command lines by hand.

What about support of rare generators? Like D++ AKA dpp, C/C++ to D headers converter tool. How many users of it in Meson? Probably, only me one.

And the same applies to any such case - C headers is "lingua franca" and looks like it is very important provide way to parse it by custom generators.

[rgommers]

We probably should have a helper for SWIG, but I don't know if we have anyone using SWIG in meson yet

xref gh-341; having to support SWIG is how I found this issue.

What about support of rare generators? ... And the same applies to any such case - C headers is "lingua franca" and looks like it is very important provide way to parse it by custom generators.

I think SWIG may be common enough to support directly, but I agree with this point. The basic abstraction that quite a few generators need is to directly get at header files. Since probably the only way to get at the location for the header file is a compile check, it seems like that should be exposed somehow. Perhaps compiler.find_header() in analogy to compiler.find_library()?

That will then allow a user to use custom_generator

mylib_path = cc.find_header('mylib.h')  # returns a single path

swig = find_program('swig', required: true, native: true)
_mylib_swig = custom_target('_mylib_swig',
    command: [swig, '- python', '-o', '@OUTPUT@', '@INPUT@'] + ['-I' + mylib_path],
    ...
)