Allow specifying `protoc` path in `buf.gen.yaml` for built-in plugins

[wingsofovnia]

Currently, there is no way for the plugins built into protoc (java, cpp, python, etc) to override in buf.gen.yaml what protoc to point to, as confirmed in https://github.com/bufbuild/buf/issues/1155#issuecomment-1139712630.

My use case, which I believe isn't that uncommon, is to publish protobuf+gRPC stubs for Python using multiple version for legacy clients. While path works fine for gRPC plugins, there is no way to swap protoc for protobuf bindings.

version: v1
plugins:
  - name: python
    path: /usr/bin/protoc-3.7    # fails
    out: out/python-legacy
  - name: grpc-python-1.32       # works fine
    out: out/python-legacy

Failure: plugin python: proto: cannot parse invalid wire-format data; proto: cannot parse invalid wire-format data

There are multiple workarounds suggested in https://github.com/bufbuild/buf/issues/1155 but neither scales well for huge and already-established workspaces.

Also, in https://github.com/bufbuild/buf/issues/1155:

The solution long-term is likely that the builtin plugins are moved out of protoc, which it appears the Google team is working on, so we probably won't add anything more here.

Can't be a solution for a production-grade tool, as it simply won't work for older plugins, which is a primary use case to support legacy clients.

Neither can remote plugin execution be an all-round solution as many enterprises can't allow exposing *.proto files for remote code generation nor/and have external dependencies which cannot be easily mirrored nor official plugins have older protoc/grpc version (see https://github.com/bufbuild/buf/issues/1289). I've also heard api.buf.build may not be available in China.

[joshcarp]

Hey @wingsofovnia A possible work around is installing protoc plugins locally from https://github.com/bufbuild/plugins which then would give you the ability to specify the version of the code generator instead of just the protoc plugin

[bufdev]

This seems like something we should add as an option. My proposal is that we add:

type ExternalPluginConfigV1 struct {
  ...
  ProtocPath string `json:protoc_path,omitempty" yaml:"protoc_path,omitempty"`
}

In private/buf/bufgen/bufgen.go. Then:

• Add this to PluginConfig as well, and weave this through parsing. You'll need to make sure this is unset by extending checkPathAndStrategyUnset to also check ProtocPath.
• Add a new appprotoexec.GenerateWithProtocPath option. There is already a appprotoexec.HandlerWithProtocPath option. Weave this through - there is already handling for this in the Handler.
• Weave this through the generator type in bufgen. You'll just need to call the new appprotoexec.GenerateWithProtocPath option.
• Testing.
• Documentation.

Implementation similar to https://github.com/bufbuild/buf/issues/1557#issuecomment-1382034075.

[bufdev]

This is released into v1.14.0.