Support per-module generation config

[nmittler]

Feature

My current setup is working with the v1 config, although there is some hackery required. I was hoping that moving to config v2 would help, but it seems that I'm still stuck.

My scenario is that I have custom proto annotations and a protoc plugin. The plugin parses an annotated resource proto files and generates gRPC service protos. With my v1 configuration, I have to comment out my custom plugin on the first call to buf generate so that the annotation golang code is available for the plugin. After that I can run buf generate to finish the job. This seems to work, albeit a bit ugly.

I was hoping that v2 and the "modules" abstraction would help here. However I don't see a way to configure generation per module. Ideally, I'd have a separate configuration for the annotations (just protoc) and a separate config for the rest of the protos that uses the custom plugin among others (grpc, etc.).

Perhaps this feature already exists and I just missed the docs?

[doriable]

You are correct in that you cannot restrict plugin configurations by input in v2 and you've correctly ascertained our current guidance is to maintain a buf.gen.yaml configuration for each combination of desired plugins and inputs.

Given that there seems to be a fair amount of demand for this, we're going to discuss internally the best way to address this. In the meantime, I'm going to link a related issue with a similar request here #2432.

[satazor]

One possible solution, that perhaps is simpler to implement, is to allow multiple "jobs" per configuration file, similar to how GitHub workflows supports one or more jobs.

Each "job" is what we have currently as the contents of the v2 format, but the v3 would allow defining multiple via an array. Example:

version: 3
jobs:
  - name: foo
    managed:
      enable: true
    plugins:
      -
    inputs:
      - directory: .

  - name: bar
    managed:
      enable: true
    plugins:
      -
    inputs:
      - git_repo: ...

Please replace jobs with something better.

This would not only solve defining a target dir for a certain input but also many other cases. The name could be a way to limit running that specific "job" when calling buf generate via a flag. By default, without any flag, buf would run all "jobs".

The only downside is that we would have a little bit of repetition, but repetition is usually better than overcomplicated overriding logic. One could say that this use-case is already supported via multiple configuration files and running buf against those, which is true. The up-side is that we wouldn't have different configuration files scattered in the project and we could run buf generate once.