Generator.process crashes when using results of another generator

[mon]

Describe the bug Trying to chain generators raises an AssertionError. Patching this code then produces an invalid ninja.build

To Reproduce For this toy example, all that is required is this meson.build and a blank file, blank.txt.

project('gentest', 'c')

cat = find_program('cat')
copy = find_program('cp')

gen1 = generator(copy, arguments: ['@INPUT@', '@OUTPUT@'], output: '@BASENAME@.g1')
gen2 = generator(copy, arguments: ['@INPUT@', '@OUTPUT@'], output: '@BASENAME@.g2')

# works OK
res1 = gen1.process('blank.txt')
# no longer OK
res2 = gen2.process(res1)
# let's use the generated result somewhere
custom_target(
    output: 'b.out',
    input: [res1, res2],
    command: [cat, '@INPUT@'],
    capture: true,
    build_by_default: true,
)

output

Traceback (most recent call last):                                                                  
  File "/home/mon/meson/mesonbuild/mesonmain.py", line 138, in run
    return options.run_func(options)
  File "/home/mon/meson/mesonbuild/msetup.py", line 294, in run
    app.generate()
  File "/home/mon/meson/mesonbuild/msetup.py", line 185, in generate
    self._generate(env)
  File "/home/mon/meson/mesonbuild/msetup.py", line 247, in _generate
    intr.backend.generate()
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 553, in generate
    self.generate_target(t)
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 725, in generate_target
    self.generate_custom_target(target)
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 986, in generate_custom_target
    self.custom_target_generator_inputs(target)
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 972, in custom_target_generator_inputs
    self.generate_genlist_for_target(s, target)
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 2242, in generate_genlist_for_target
    elem.add_dep([self.get_target_filename(x) for x in generator.depends])
  File "/home/mon/meson/mesonbuild/backend/ninjabackend.py", line 2242, in <listcomp>
    elem.add_dep([self.get_target_filename(x) for x in generator.depends])
  File "/home/mon/meson/mesonbuild/backend/backends.py", line 319, in get_target_filename
    assert isinstance(t, build.BuildTarget)
AssertionError

Modifying the code here so that GeneratedList is treated identically to CustomTarget starts producing invalid ninja depedencies:

ninja: error: 'blank.g1', needed by 'b.out.p/blank.g2', missing and no known rule to make it

I've got halfway to a solution using a new sentinel type which is put into the target's private directory during ninja.build generation, but it feels like a very heavy solution. I am dedicated to fixing it but would like some guidance on easier strategies.

system parameters

• 0.60.99
• master at time of submission - 95a4c6a62a925fc888e7c07ae3d9937c87e1c759
• Ubuntu on WSL
• Python 3.8.10
• ninja 1.10.0

[ArthurHeymans]

I have the same issue. I need to preprocessor nasm files before assembling them. What is the status on this?

[smallwood-d]

The issue seem to be resolved in master branch.

system parameters

• meson version 0.63.99
• master at time of commenting - https://github.com/mesonbuild/meson/commit/0ec039d2597526c670f7e0478a7b5008e14ff474
• Ubuntu 22.04 on WSL
• Python 3.10.4
• ninja 1.10.1

while the current meson version (installed using apt) didn't fix the issue.

• meson version 0.61.2

[dcbaker]

0.61.2 is fairly old, can put try something from 0.62 or 0.63 (current)?

[eli-schwartz]

git bisect says that this reproducer stopped crashing with commit d0a0e04c987ea92473073f9c3017fe648fc59f04. (Released in 0.63.0)

Congrats @dcbaker, you fixed it. :)

EDIT: No, it's not fixed, building results in the error mentioned above:

$ ninja
ninja: error: 'blank.g1', needed by 'b.out.p/blank.g2', missing and no known rule to make it

[dcbaker]

I’ll look at it, probably isn’t too hard to fix

[smallwood-d]

If issue is not a high priority me and @duckflyer will be happy to try and work on it.

[dcbaker]

Oof, this issue is actually a lot harder than it looks. We basically assume that a Generator output is part of a target, and generate the output directory for the generator in the backend based on the conusming Target.

[vinipsmaker]

I'm also affected by the issue. I need to use two code generators. The first one is re2c that will generate lexers. Once I have the output for that, I pass it to a second generator (gperf). However the issue I get from the command line is very different from the initial report for this issue:

ninja: error: 'actor.cpp', needed by 'emilua.p/bc2_actor.cpp', missing and no known rule to make it

File src/actor.ypp was fed to re2c to generate actor.cpp. actor.cpp should be used as input in gperf.