protobuf-maven-plugin doesn't support protoc insertions

[crazyk2]

I use maven and faced with a problem when had used your plugin. Short log of problem is [ERROR] PROTOC FAILED: some/path//SomeClass.java: Tried to insert into file that doesn't exist. My pom.xml

<plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}</protocArtifact>
                    <clearOutputDirectory>false</clearOutputDirectory>
                    <outputDirectory>${basedir}/target/generated-sources/protobuf/java/</outputDirectory>
                </configuration>
                <executions>
                    <execution>
                        <goals><goal>compile</goal></goals>
                    </execution>
                    <execution>
                        <id>grpc-java</id>
                        <goals><goal>compile-custom</goal></goals>
                        <configuration>
                            <pluginId>grpc-java</pluginId>
                            <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.18.0:exe:${os.detected.classifier}</pluginArtifact>
                        </configuration>
                    </execution>
                    <execution>
                        <id>kroto-plus</id>
                        <goals>
                            <goal>compile-custom</goal>
                        </goals>
                        <configuration>
                            <pluginId>kroto-plus</pluginId>
                            <pluginArtifact>com.github.marcoferrer.krotoplus:protoc-gen-kroto-plus:${krotoplus.version}:jar:jvm8</pluginArtifact>
                            <pluginParameter>ConfigPath=${project.basedir}/krotoPlusConfig.asciipb</pluginParameter>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

as noticed there, there and there

Using dsl markers relies on protoc insertions, so take care to ensure that the kroto-plus output directory is the same as the directory for generated java code

Then I tried protobuf-gradle-plugin with settings

protobuf {
    protoc { artifact = "com.google.protobuf:protoc:${versions.protobuf}" }
    generatedFilesBaseDir = "$buildDir/generated-sources"
    plugins {
        grpc { artifact = "io.grpc:protoc-gen-grpc-java:${versions.grpc}" }
        kroto {
            artifact = "com.github.marcoferrer.krotoplus:protoc-gen-kroto-plus:${versions.krotoplus}:jvm8@jar"
        }
    }
    generateProtoTasks {
        def krotoConfig = file("krotoPlusConfig.asciipb")
        all().each { task ->
            task.inputs.files krotoConfig
            task.plugins {
                grpc { outputSubDir = "java" }
                kroto {
                    outputSubDir = "java"
                    option "ConfigPath=$krotoConfig"
                }
            }
        }
    }
}

and that worked. I ran command in debug mode and compare output protobuf-maven-plugin and protobuf-gradle-plugin. protobuf-gradle-plugin run only one protoc command with pipe of plugins

protoc-3.6.1-linux-x86_64.exe -I/api/src/main/proto -I/api/build/extracted-protos/main -I/api/build/extracted-include-protos/main --java_out=/api/build/generated-sources/main/java --plugin=protoc-gen-grpc=/home/stepan/.gradle/caches/modules-2/files-2.1/io.grpc/protoc-gen-grpc-java/1.16.1/f26e45fd6dc4ebaf6d76b22b290b337ee96aed54/protoc-gen-grpc-java-1.16.1-linux-x86_64.exe --grpc_out=/api/build/generated-sources/main/java --plugin=protoc-gen-kroto=/home/stepan/.gradle/caches/modules-2/files-2.1/com.github.marcoferrer.krotoplus/protoc-gen-kroto-plus/0.2.2-SNAPSHOT/9c02fa688b3c00b1f768fb9302e85205a798fc05/protoc-gen-kroto-plus-0.2.2-SNAPSHOT-jvm8.jar --kroto_out=ConfigPath=/api/krotoPlusConfig.asciipb:/api/build/generated-sources/main/java some_protos.proto ...

--java_out --> --protoc-gen-grpc --> protoc-gen-kroto protobuf-maven-plugin run protoc commands N executions times and in my case last command is

protoc-3.6.1-linux-x86_64.exe --proto_path=/api/src/main/proto --proto_path=/api/target/protoc-dependencies/3f38729cce82adb9f25c7327ca2059db --proto_path=/api/target/protoc-dependencies/5f1e9201db2bd80331b59aeb26f3d508 --plugin=protoc-gen-kroto-plus=/api/target/protoc-plugins/protoc-gen-kroto-plus-0.2.2-SNAPSHOT-jvm8.jar --kroto-plus_out=ConfigPath=/api/krotoPlusConfig.asciipb:/api/target/generated-sources/protobuf/java some_protos.proto ...

that lead to build failure. When I have added --java_out=/api/target/generated-sources/protobuf/java to command above. It worked fine

protoc-3.6.1-linux-x86_64.exe --proto_path=/api/src/main/proto --proto_path=/api/target/protoc-dependencies/3f38729cce82adb9f25c7327ca2059db --proto_path=/api/target/protoc-dependencies/5f1e9201db2bd80331b59aeb26f3d508 --java_out=/api/target/generated-sources/protobuf/java --plugin=protoc-gen-kroto-plus=/api/target/protoc-plugins/protoc-gen-kroto-plus-0.2.2-SNAPSHOT-jvm8.jar --kroto-plus_out=ConfigPath=/api/krotoPlusConfig.asciipb:/api/target/generated-sources/protobuf/java some_protos.proto ...

I don't know which products this bug\behavior belongs to.

[marcoferrer]

I spent some time digging into this. You're 100% correct about the lack of --java_out. It looks like omitting it causes issues with the protoc insertions api.

It also looks like the protobuf maven plugin invokes protoc once for each plugin defined. The --java_out arg is not added when configuring custom native plugins (as seen here) but is added when using java plugins (as seen here).

I tried using the java plugin configuration since it seems to be the only available option for including java out arg.

<plugin>
    <groupId>org.xolstice.maven.plugins</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <extensions>true</extensions>
    <version>0.6.1</version>
    <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}</protocArtifact>
        <outputDirectory>${basedir}/target/generated-sources/protobuf/java</outputDirectory>
        <clearOutputDirectory>false</clearOutputDirectory>
        <protocPlugins>
            <plugin>
                <id>kroto-plus</id>
                <groupId>com.github.marcoferrer.krotoplus</groupId>
                <artifactId>protoc-gen-kroto-plus</artifactId>
                <version>${krotoplus.version}</version>
                <classifier>jvm8</classifier>
                <mainClass>org.springframework.boot.loader.JarLauncher</mainClass>
            </plugin>
        </protocPlugins>
    </configuration>
    <executions>
        ...
    </executions>
</plugin>       

I checked out the output from this configuration and it does indeed properly pass the java out arg BUT now theres a new issue. The protobuf maven plugin doesnt allow you to pass any pluginParameter arguments when using this style of configuration.

I tried every supported configuration with no success. This looks like a bug in the maven plugin. It doesnt support the insertions api using custom native plugins. The insertions api should work in theory if you use a java plugin but doesn't support passing pluginParameter arguments.

[auzhva]

Still doesn't work... Not really usable with maven :(