Add support for using proto artifact dependencies

[aantono]

Similar to https://github.com/google/protobuf-gradle-plugin#protos-in-dependencies add ability to reference artifact dependencies (in the form of tar.gz or zip archives) to be used as sources of *.proto files.

[sergei-ivanov]

Hi, This is already supported for jar dependencies and reactor modules. *.proto files from inside dependent projects are automatically unpacked and added to import path for protoc.

[fzakaria]

I'm trying something like

    <dependencies>
        <dependency>
            <groupId>test</groupId>
            <artifactId>model</artifactId>
            <version>1.0-SNAPSHOT</version>
            <classifier>proto</classifier>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.5.0</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>

                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Doesn't seem to generate anything

[sergei-ivanov]

If your project itself does not contain any proto sources and all proto sources are instead sitting inside the dependency artifact, then nothing will be compiled. You will have to pull the artifact down and unpack it into your build directory using maven-dependency-plugin, and then use that as a source directory for protobuf-maven-plugin.

[aantono]

Any way that can be done automatically by the plugin instead?

[sergei-ivanov]

No. It would be pretty un-maven, because it'd unnecessarily overcomplicate the plugin's logic and configuration (currently, people pull dependency artifacts to use as imports, not to compile them). If you look at other maven plugins (maven-compiler-plugin, maven-jar-plugin, maven-source-plugin), they only work with the files inside the local project directory. There's a good reason for it: they adhere to 'convention over configuration' principle and try to do one thing and do it right. There are other plugins that can be added to the build lifecycle in order to handle unconventional configurations. Please note that the gradle plugin that you linked to explicitly makes the same statement: The imported proto files will not be compiled since they have already been compiled in their own projects. I could have added a separate goal to pull down another artifact with proto sources, but that is pretty much already covered by maven-dependency-plugin, so there's not much point in me replicating that functionality.

[fzakaria]

btw --

If there is a project, package or published artifact that contains just protos files, whose compiled classes are absent, and you want to use these proto files in your project and compile them, you can add it to protobuf dependencies. Example:

dependencies {
  protobuf files('lib/protos.tar.gz')
  testProtobuf 'com.example:published-protos:1.0.0'
}

They create a separate dependecy type to compile proto only dependencies Perhaps if they adhere to a classifier it seems that it would follow "convention" or something else -- but w/e.

[luigi-riefolo]

@sergei-ivanov could you please provide an example on how to import a proto that has been defined in a different project? i.e. There are two projects, A and B. B has already been compiled and packaged in a jar. While project A has a dependency on a protobuf message specified in project B. How does project A declare its dependency on project B? And how does the notation look like in the proto file? Thanks.

[sergei-ivanov]

@luigi-riefolo you can browse through integration tests (src/it/...), there is plenty of examples there, with reactor projects and 3rd party dependencies. In particular, this test is a perfect example that also demonstrates how packages are resolved.

The main requirement is that if a protobuf package is declared in the proto file, it must match its subdirectory under src/main/proto. If there is no package declaration, the file must reside directly inside src/main/proto and not in any subdirectory thereof. Otherwise there will be trouble.

So if you have in your test1.proto file:

package it.project1;

... then the proto file should reside in src/main/proto/it/project1. And another proto file (possibly in another Maven module or project) will then be able to import it:

import "it/project1/test1.proto";

I hope that helps.

[prometheus-chained]

I would like to use same proto file for client and server generation. Suggested approach does not work because:

1. It still requires create proto file otherwise plugin would skip execution.
2. Even if I create proto file with single import protoc will not generate any DTO's or server/client stub's.

The only working way to share proto file between server and client is to unpack it to some directory and add it as protoSourceRoot.

Why it is a problems to take proto from dependencies? Plugin already take proto files from dependencies and unpack them to target dir, why it is not possible to use those proto files as main proto definition?