Arshia001 / FSharp.GrpcCodeGenerator

A protoc plugin to enable generation of F# code + supporting libraries
MIT License
81 stars 9 forks source link
code-generation fsharp grpc protobuf protobuf-message protocol-buffers

FSharp.GrpcCodeGenerator

This project is aimed at providing complete integration of gRPC and protocol buffers into the F# language. The project is early in development and, while all features have been implemented, it's not guranteed to work correctly.

Suggestions, bug reports and pull requests are very welcome.

How do I use it?

How does it work?

Now that that's out of the way, let's see how it all works.

Messages

Protobuf messages are transformed into F# records. This means we get to use F#'s automatic implementations for Equals, ToString etc. Now, you may be used to strictly immutable records, so be warned: the records contain mutable fields. This is because the .NET protobuf runtime is implemented with mutability in mind, and object are constructed and initialized in separate phases. It'd be a much bigger task to adapt the runtime to strictly immutable messages.

Fields

All primitive and message fields are transformed into ValueOption fields. This is because any field can simply be missing from a protobuf message, and we don't want to start guessing whether a 0 came in the actual message or was filled in as a default.

On the plus side, we no longer need HasXXX and ClearXXX calls. Simply match the ValueOption to test for a field's presence and set it to ValueNone to clear it from the message. Another benefit is that the deserializer never generates null fields; if you see one, it's a bug that needs to be reported.

But why use value option, I hear you ask? Performance! I wouldn't want each field of each message to cause an allocation. I know you wouldn't either. Someone could implement a switch to override this behaviour, but I won't. Allocation is evil.

And BTW, if you think handling all those options all over your code is troublesome, you need to stop passing network DTO's into your business logic. You may want to take a look at Onion architecture.

Collections

Repeated and map fields use the built-in RepeatedField and MapField types from protobuf, which include special support for the binary protocol.

Enums

Enums are transformed into CLR enums, not F# unions. This is due to the fact that an enum field is, in the end, a number field, which is a perfect match for CLR enums, including the possibility of encountering numbers with no assigned name.

One-of

One-of fields, however, are transformed into F# union types. Contrary to the C# plugin which generates separate properties for each one-of entry, you'll get only one record field of type ValueOption<UnionType>. A ValueNone indicates none of the fields were present in the message. If one was present, you'll get a ValueSome(UnionCase). If more than one is present (and you have a malformed message), only the last value will be preserved.

Helpers

Each message gets an accompanying module. It contains:

Reflection

Each file also gets a Reflection module. This module contains the reflection descriptor for the whole file. It also contains descriptors for each message type, should you need to access them by name.

Services

Services are transformed into classes. This is due to the OO nature of the runtime. For each service, you get a client class and an abstract server base class.

You also get a MyService.MyServiceClient.Functions module with F# functions that take an instance of the client class as input. This is meant to facilitate function composition, since composing instance methods (when possible at all) is kind of a nightmare...

The one divergence from the C# code generator is that the server base class contains no default logic to return a "not implemented" response; you get abstract methods instead.

What else?

Show me some code

Here you are. First, reading and writing messages directly:

open Google.Protobuf

// Read a message
use stdIn = Console.OpenStandardInput()
let req = Compiler.CodeGeneratorRequest.Parser.ParseFrom(stdIn)

// Do something with it
let files = ... // left out

// Create a response message
let resp = { Compiler.CodeGeneratorResponse.empty() with SupportedFeatures = ValueSome <| uint64 Compiler.CodeGeneratorResponse.Types.Feature.Proto3Optional }
resp.File.AddRange files

// Write it somewhere
use stdOut = Console.OpenStandardOutput()
// WriteTo is provided in Google.Protobuf.MessageExtensions
resp.WriteTo(stdOut)

Here's a service client:

use channel = GrpcChannel.ForAddress("https://localhost:5001/")
let client = Greet.GreeterService.GreeterServiceClient(channel)
let req = { Greet.HelloRequest.empty() with Name = ValueSome "World" }
let resp = client.SayHello(req).ResponseAsync.Result
printfn "%s" resp.Message

And a service implementation:

type GreeterService() =
    inherit Greet.GreeterService.GreeterServiceBase()

    override _.SayHello req ctx =
        let resp =
            { Greet.HelloReply.empty() with
                // Notice how we're immediately forced to handle missing fields.
                // The language itself protects you from the binary protocol's quirks.
                // How cool is THAT?
                Message = req.Name |> ValueOption.map (sprintf "Hello, %s!")
            }
        Threading.Tasks.Task.FromResult(resp)

A note on protoc plugins

It took me considerable effort (much more than the ~5 minutes it takes to read the following paragraphs) to figure out how to implement a protoc plugin. If there is adequate documentation anywhere, I must have missed it; so I'll document what I've learned here.

The protocol buffers compiler supports plugins in the shape of executables named protoc-gen-XXXX, where XXXX is the name of the plugin. You enable a plugin by specifying --XXXX_out=some_directory and optionally pass it arguments by specifying --XXXX_opt=opt1=val1,opt2=val2. protoc attempts to execute protoc-gen-XXXX and write a serialized code generation request to the process's stdin. It then waits for the process to write a code generation response back to its stdout. The request and response are serialized as (you guessed it!) a protobuf message. The contract for these types is available from google/protobuf/compiler/plugin.proto, found inside the protoc download archive.

(Fascinatingly, the C# implementation inside protoc does not handle GRPC services. Those are generated by a separate plugin. This is why in C# you get two source files per .proto file instead of one.)

If you're implementing a plugin in C++, you can use a bunch of utilities the protobuf team have made available, but if you're implementing a plugin in another language, you'll have to be able to understand protobuf in order to understand protobuf, so to speak. This chicken-and-egg situation is the same as with any language which has its primary compiler implemented in the language itself. You'd basically have to implement the initial version in another language and then use that to implement the language again, this time in itself. I was lucky enough to have the C# protobuf plugin available, which means I implemented the original code in F#, and only had to account for the change from classes to records and such minor cases. It may not be as easy for another language where no support is available.