[Question] Can a meson.build recipe generate a wrap file to be used later?

[gabeblack]

I have a .wrap file that specifies which project to retrieve from a third-party and I use packagefiles to house the meson.build file that will then build the project (they use Makefile). However, the project has its own custom "dependency" file where its own third-party build-time dependencies are enumerated.

I tried writing a script that will generate wrap files and a simple meson.build file for those '4th-party' packages and put it in a subprojects + subprojects/packagefiles within the downloaded subproject folder.

So essentially it is:

subprojects/packagefiles/foo/meson.build

project('foo', 'c')

run_command('generate_my_dependency_wrap_files.py', 'bar1', 'bar2', 'bar3')

subproject('bar1')
subproject('bar2')
subproject('bar3')

The run_command('generate_my_dependency_wrap_files.py' ...) above will generate:

• subprojects/bar1/subprojects/bar1.wrap
• subprojects/bar1/subprojects/packagefiles/bar1/meson.build
• subprojects/bar2/subprojects/bar2.wrap
• ... (etc)

The first time that meson setup is run, it says that 'wrap-redirect bar1.wrap filename does not exist'. But if you run it again, it will succeed (because run_command has been invoked by then). So, long story short, is there a way to generate wrap files that can then be used without having to run meson setup twice? The reason to do something like this, is that third-party project 'foo' contains the revision information that would go in a wrap file for the dependencies it needs -- I don't have this information outside of project foo.

[eli-schwartz]

run_command is "unsafe". It allows you to do literally anything, including run a python script that modifies the meson.build containing the run_command in order to remove it.

Meson will not guarantee any behavior if you modify the build files while it is running. With great power (to run arbitrary commands to fetch data) comes great responsibility (to avoid using it for anything other than fetching data).

One possibility would be to have one wrap file per revision, without doing generation, and simply checking in project "foo" which wrap to use. e.g.

subproject('bar1-v'+bar1_version)

[gabeblack]

That would require having to know about my third-party dependency's dependencies ahead of time. That isn't the end of the world. I probably can instead make the run_command verify that the correct '4th-party' dependencies are being used and error-out if they are not (informing the user to update the wrap files). That's very do-able.

Thanks! BTW, I have been porting a very large company product to meson that includes a linux kernel build, various custom kernel modules, java generators, mixed build systems (cmake, autotools, custom), cross-compilers, compiling source code generators and then using them to generate other source files, distributed building (dmucs + distcc), and now this. I'm about 75% done and build time has gone from 3-5 hours (depending on options) to ~5-20 minutes. Also, no-change rebuild is instantaneous, where a no-change rebuild was 45 minutes in the old system. So just wanted to say THANK YOU to the meson project!

[eli-schwartz]

That would require having to know about my third-party dependency's dependencies ahead of time. That isn't the end of the world. I probably can instead make the run_command verify that the correct '4th-party' dependencies are being used and error-out if they are not (informing the user to update the wrap files). That's very do-able.

Yup, at the very least this would make it evident to users that they need to rerun meson. :)

Thanks! BTW, I have been porting a very large company product to meson that includes a linux kernel build, various custom kernel modules, java generators, mixed build systems (cmake, autotools, custom), cross-compilers, compiling source code generators and then using them to generate other source files, distributed building (dmucs + distcc), and now this. I'm about 75% done and build time has gone from 3-5 hours (depending on options) to ~5-20 minutes. Also, no-change rebuild is instantaneous, where a no-change rebuild was 45 minutes in the old system. So just wanted to say THANK YOU to the meson project!

Awesome! That's wonderful to hear.

By the way this does remind me of @jpakkane's interest in #10690.