Proposal: Introduce `bal protoc` command to generate source code from .proto files

[MadhukaHarith92]

Summary

The official tool that generates source files from .proto files is protoc. Need to introduce a Ballerina tool that is compatible with protoc.

Goals

Ballerina protobuf generation tool should be in sync with other programming languages such as python and go.

Motivation

To deprecate the bal grpc command and introduce the bal protoc command which is more compatible with protoc.

Description

The current bal grpc command generates source code from a .proto file. The source code includes messages and message encode/decode related code and gRPC client codes. However, this command is not accurate since it combines protobuf with gRPC which are two different concepts.

Other programming languages such as go and python have protoc commands to generate source code from .protoc files. This only generates messages and messages encode/decode related code. The grpc is a plugin on top of that which generates gRPC-related code. For instance,

protoc -I product_info.proto --go_out=plugins=grpc:go/server/ecommerce

In the above command, the grpc is defined as a plugin. We need to introduce a similar command to generate source code from .proto files.

Command: bal protoc

Flags: --out | -o - Generated Ballerina source files location with messages and message encode/decode related code

--grpc-out | -g - Generated Ballerina source files location with messages and message encode/decode related code and the services-related code

--imports | -i - Path to a directory in which to look for .proto files when resolving import directives

--grpc-opt - Options to be passed when generating messages and message encode/decode related code and the services-related code

Currently, only mode is a possible option. User can pass --grpc-opt=mode=service to generate sample service source code or --grpc-opt=mode=client to generate sample client source code.

Example: Executing the following command will generate messages and message encode/decode related code.

bal protoc --out go/server/e-commerce hello.proto

Executing the following command will generate messages and message encode/decode related code and the services-related code.

bal protoc --grpc-out go/server/e-commerce --grpc-opt=mode=service hello.proto 

Testing

• Execute the command with --out flag. Verify that only the messages and message encode/decode related code are generated.
• Execute the command with the --grpc-out flag. Verify that messages and messages encode/decode related code and the services-related code are generated.
• Assert the content of the generated commands.
• Assert that the generated source code do not contain any semantic errors or syntax errors.

[daneshk]

The protoc command is used for the protobuf, which only handles messages and message encode/decode related code generation. grpc is a plugin on top of it. better, we look into that aspect as well.

e.g: in golang,

protoc -I product_info.proto --go_out=plugins=grpc:go/server/ecommerce

[daneshk]

The latest golang protoc tool commands have been changed. please check the doc[1]

Shall we follow the same approach for the new protoc commands? like,

bal protoc -i --grpc-out go/server/e-commerce --grpc-opt=mode=service hello.proto

Let's list down all parameters.

1. https://grpc.io/docs/languages/go/quickstart/

[daneshk]

LGTM. For the first iteration, we can omit --out parameter as we do not support to use protoc for other protocols. We can introduce once we have the support

[daneshk]

@sameerajayasoma @shafreenAnfar can you check and approve

[sameerajayasoma]

I also suggested this sometime back. But now I have a different view on this.

In the context of Ballerina, this tool generates the Ballerina gRPC service and the client object. By default, it uses the protobuf encoding. AFAIK, gRPC is not tied to protobuf. See: https://grpc.io/blog/grpc-with-json/

Since the main focus of this tool is to generate Ballerina gRPC services and clients, the current tool name is correct. Let's introduce an option to change the default encoding/serialization to JSON or other serializations.

[sameerajayasoma]

I want to suggest some changes to bal grpc command signature to make it consistent with other bal commands.

Most of the sub-commands in the bal tool use the following convention:

bal <subcommand> <input>

The default output location depends on the particular command, and the location can be customized with the -o <output>, --output <output> option.

For example: 1) bal grpc demo.proto

• If the command is executed inside the package root, copy the generated sources into the generated directory.

I will create another issue with the complete suggestion.