Feature Request: Optional Message Prefix for code generation (for improved Kotlin/Native support)

[darronschall]

I'm using PBandK to generate the networking layer in a Kotlin Multiplatform Mobile project (iOS/Android apps) with the help of a service-gen plugin I open-sourced at https://github.com/collectiveidea/twirp-kmm.

Due to the lack of packages in Objective-C, Kotlin/Native renames classes that share the same name by appending underscores to avoid name collisions. This presents issues when protobuf message classes share the same class name as client-side models. For example, if protobuf defines an example.api.Item message but there also exists an example.model.Item model class, Kotlin/Native renames one of them to Item_ so that both classes can be used in Swift code. Working with Item_ and Item is both confusing and unpleasant. There is a feature request at https://youtrack.jetbrains.com/issue/KT-50767/Kotlin-Native-Objective-C-Stable-rename-algorithm-for-classes-with-same-name to improve Kotlin/Native support for this scenario, which suggests a new @SwiftName annotation to improve interop, but no timeframe is given on implementing it.

Ideally, PBandK would allow an option to append a suffix to protobuf messages during the generation process. Something like message_suffix (commonly DTO in this context):

protoc --pbandk_out=message_suffix=DTO,kotlin_package=com.example.api:shared/src/commonMain/kotlin sample.proto

This would cause the protobuf message Example to be generated as @pbandk.Export public data class ExampleDTO. The suffix would not apply to messages ending in Request or Response. In the future, should Kotlin/Native introduce @SwiftName per the linked Youtrack, then the suffix option might instead generate @pbandk.Export @SwiftName("ExampleDTO") public data class Example. EDIT: Kotlin/Native 1.8.0 adds a @ObjCName annotation that the generated code can leverage.

This is a problem that I needed to solve in the short-term. As a workaround, I've created a small bash script that pre-processes my project's protobuf file to append the DTO suffix to messages before running the PBandK code-gen. The net result is the that the generated code appends DTO to the message class names. My (macOS) script looks roughly like this (for anyone else running into the same issue):

#
# Pre-process the `example.proto` file to change the message names to have a DTO suffix.
#

# First, copy the example.proto file to example_processed.proto, where we'll make our edits.
cp shared/src/commonMain/proto/example.proto example_processed.proto

# Process the file extract a list of message class names. Use `pcre2grep` to easily capture the group
# within the matching regex, to isolate the type name
pcre2grep -o1 "^message ([A-Za-z0-9]+) {" example_processed.proto | \

# From that this list, strip out type names ending in either `Request` or `Response` since
# we don't want to append the `DTO` suffix to those.
 grep -v "Response$" | grep -v "Request$" | \

# At this point, our list looks something like:
#    Item
#    Message
#    Category
# (etc.)

# Replace the message type names with the type name followed by the DTO suffix. We do this by
# reading all of the types from our stdin list and replacing them in the target file.
while read -r type; do
  # The type names are either surrounded by parens or spaces; replace both.
  sed -i '' -e "s/($type)/(${type}DTO)/g" -e "s/ $type / ${type}DTO /g" example_processed.proto
done

# Generate the protobuf code from the processed file
protoc --pbandk_out=kotlin_package=com.example.api:shared/src/commonMain/kotlin example_processed.proto

# The generated code is placed in `example_processed.kt`. We need to move that to the correct
# location, overwriting the `ExampleProtobufs.kt` file if it already exists.
mv -f shared/src/commonMain/kotlin/com/example/api/example_processed.kt shared/src/commonMain/kotlin/com/example/api/ExampleProtobufs.kt

# Delete the processed proto file now that we're done with it.
rm example_processed.proto

[garyp]

Now that Kotlin 1.8 added the @ObjCName annotation, I'm thinking the best solution would be for pbandk to read the objc_class_prefix and swift_prefix options (see https://github.com/protocolbuffers/protobuf/blob/b4811c3ffb6c7d259e5771822918db8a1eb52f7f/src/google/protobuf/descriptor.proto#L429 for the option definitions) from the proto file and translate those into appropriate @ObjCName.name and @ObjCName.swiftName annotations.

We can now also have pbandk use the @ObjCName annotation to implement the default behavior documented for the swift_prefix option when the option is not explicitly provided:

// By default Swift generators will take the proto package and CamelCase it // replacing '.' with underscore and use that to prefix the types/symbols // defined.

@darronschall would the above two changes address your use case?

[darronschall]

@garyp Nice suggestion, that solves the problem elegantly. Thanks!

[garyp]

I've got some other large pbandk changes in the queue, so I won't be able to work on this soon. Pull requests are always welcome though 😃 I think this should be a relatively straightforward change in CodeGenerator to add the new annotations. I'm happy to provide pointers.

[darronschall]

Thanks @garyp! I took a look at CodeGenerator, and while I'm not familiar with this code base I can see where the code should be modified to output the Kotlin 1.8 annotations.

The biggest question I have off-hand is: Do I need to do anything special to grab the swift_prefix or objc_class_prefix values form the proto file? I see these are FieldDescriptors here and here. I'm assuming this is pass-through from protoc and I don't need to do any parsing, but I'm unclear at the moment on how I would access the values within CodeGenerator. Possibly from File? But nothing there is jumping out at me either.

If you know the answer off the top of your head, great! If not, I can dive in when I get some time and play around with it.

[garyp]

@darronschall You'll need to add new fields to that File type to contain the Swift and ObjC prefixes. Take a look at FileBuilder.buildFile() (https://github.com/streem/pbandk/blob/f948160a396d82d890f5ce62bd262595871d1e40/protoc-gen-pbandk/lib/src/commonMain/kotlin/pbandk/gen/FileBuilder.kt#L10) to see how the File instance is constructed. You'll need to grab the values of ctx.fileDesc.options?.swiftPrefix and ctx.fileDesc.options?.objcClassPrefix.

Once you have the values added to File, you can access them from inside of CodeGenerator.