Protobuf structures are hard to import in a separate project.

[stswidwinski]

Proposal

I believe that the handling of protobuf files within Nomad it difficult to use them in a separate project. This seems quite acute for anything which would want to reuse the existing data structures in the extensions supported by Nomad (such as the task driver or the device driver).

One example would be a project which implements a Task Driver, which wants to extend and reuse pieces of the Nomad-provided shared executor. It would be quite natural to write something like:

syntax = "proto3";
package my.package.extending.nomad;
option go_package = "proto";

import "github.com/nomad/drivers/shared/executor/proto/executor.proto"

[...]

However, this is not possible to do without manual modifications of the Nomad repository for a few reasons:

1. buf.yaml which is currently used is hidden in tools/buf and as such does not mark the root of a module (see: https://buf.build/docs/bsr/module/manage)
2. The existing buf being used in Nomad is of version 0.36.0 which is quite out of date and uses v1beta1 version which is ill-documented and hard to reason about
3. Nomad protos are not available in BSR (see: https://buf.build/docs/bsr/introduction) which means that linking against protos requires cloning of the nomad codebase and manual munging of the directory tree

It would be great if:

1. The buf usage could be modernized to support v1 rather than v1beta1
2. The protos be uploaded into BSR and versioned according to releases

Use-cases

Any application which attempts to use or extend the existing protobufs of Nomad

Attempted Solutions

There exist work-arounds which are primarily centered around manual munging of the directory structure.

[jrasell]

Linked to a previous issue which discusses problems with updating Buf: https://github.com/hashicorp/nomad/issues/10293

[tgross]

Thanks for that context @jrasell. I'm going to mark this one for roadmapping, as it's definitely something we want to get fixed at some point.