search

xolstice / protobuf-maven-plugin

Maven Plugin that executes the Protocol Buffers (protoc) compiler
https://www.xolstice.org/protobuf-maven-plugin/
Other
232 stars 76 forks source link

Protos in non-default folder #36

Closed magick93 closed 6 years ago

magick93 commented 6 years ago

hello

I am trying to generate using protos that are not in the default folder.

The basic summary of the problem is: When I have:

<configuration>
                    <clearOutputDirectory>false</clearOutputDirectory>
                    <protoSourceRoot>${basedir}/src/main/proto/my-protos/</protoSourceRoot>
                    <includes>
                        <include>${basedir}/src/main/proto/my-protos/*.proto</include>
                    </includes>
                     <checkStaleness>false</checkStaleness>
                    <protocArtifact>com.google.protobuf:protoc:3.4.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.6.1:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>

Then I get:

[INFO] No proto files to compile.

However, if I have:

<configuration>
                    <clearOutputDirectory>false</clearOutputDirectory>
                    <protoSourceRoot>${basedir}/src/main/proto/my-protos/</protoSourceRoot>
                    <!-- <includes>
                        <include>${basedir}/src/main/proto/my-protos/*.proto</include>
                    </includes> -->
                     <checkStaleness>false</checkStaleness>
                    <protocArtifact>com.google.protobuf:protoc:3.4.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.6.1:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>

Then it fails to find messages, and complains about messages not being found.

When I do so. the maven plugin cannot find related proto files that are in the same directory. If I specify the location of the proto files using protoSourceRoot directive, it makes no difference.

When I specify the folder using the Include directive, the generator does not error, but does not generate anything.

org.xolstice.maven.plugins protobuf-maven-plugin 0.5.0 false /home/anton/git/full/path/to-my-example/src/main/proto/my-protos com.google.protobuf:protoc:3.4.0:exe:${os.detected.classifier} grpc-java io.grpc:protoc-gen-grpc-java:1.6.1:exe:${os.detected.classifier} compile compile-custom ``` [INFO] ------------------------------------------------------------------------ [INFO] Building io.grpc:grpc-netty 1.6.1 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- protobuf-maven-plugin:0.5.0:compile (default) @ artifactid --- [INFO] Compiling 3 proto file(s) to /home/anton/git/priceinsight/tmp/mt4manager-client/target/generated-sources/protobuf/java [ERROR] PROTOC FAILED: UsersRequestRequestMessage.proto: File not found. UsersService.proto: Import "UsersRequestRequestMessage.proto" was not found or had errors. UsersService.proto:11:27: "UsersRequestRequestMessage" is not defined. [ERROR] /home/anton/git/priceinsight/tmp/mt4manager-client/src/main/proto/my-protos/UsersService.proto [0:0]: UsersRequestRequestMessage.proto: File not found. UsersService.proto: Import "UsersRequestRequestMessage.proto" was not found or had errors. UsersService.proto:11:27: "UsersRequestRequestMessage" is not defined. [ERROR] /home/anton/git/priceinsight/tmp/mt4manager-client/src/main/proto/my-protos/UsersRequestRequestMessages.proto [0:0]: UsersRequestRequestMessage.proto: File not found. UsersService.proto: Import "UsersRequestRequestMessage.proto" was not found or had errors. UsersService.proto:11:27: "UsersRequestRequestMessage" is not defined. [ERROR] /home/anton/git/priceinsight/tmp/mt4manager-client/src/main/proto/my-protos/UsersRequestResponseMessage.proto [0:0]: UsersRequestRequestMessage.proto: File not found. UsersService.proto: Import "UsersRequestRequestMessage.proto" was not found or had errors. UsersService.proto:11:27: "UsersRequestRequestMessage" is not defined. [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.929 s [INFO] Finished at: 2017-10-25T13:06:16+13:00 [INFO] Final Memory: 10M/163M ``` I have attached a simple sample. [proto-maven-not-generating.zip](https://github.com/xolstice/protobuf-maven-plugin/files/1412970/proto-maven-not-generating.zip)
sergei-ivanov commented 6 years ago
  1. Includes/excludes lists specify file patterns. ${basedir} is an absolute path, and I suspect that by adding it into the include pattern you are effectively excluding everything, because nothing will match it. Includes/excludes are needed very seldom, only for some non-trivial source directory layouts.
  2. By going against the convention and placing your proto files in a custom root you are making your life harder for no good reason. If you are creating a new project (as opposed to converting a legacy project), you should always try to stick to maven conventions and avoid unnecessary customisations. In any case, you first need to produce a fully working configuration with your protobuf files and only then you can start customising it. Otherwise there are too many moving parts.
  3. Your proto files in the attached example use package declarations, but they are not placed in the correct directory structure according to their package. They should be sitting in <proto_root>/my/package/xxx.proto. Again, you are making your life more difficult by introducing packages, and again I advise you to get a properly working project configuration without any excessive customisations and only then start tweaking things. Otherwise you are wasting time of other people who end up doing your homework for you. I am happy to help with non-trivial problems, but most of your problems could be resolved by reading the relevant documentation and not trying to complicate things without necessity.
  4. Before opening bugs against maven plugin, please make sure your proto files can be compiled by protoc on the command line. If you run your maven command with -X, it will output a lot of useful debug information that might be helpful in figuring out what is being passed to protoc.
magick93 commented 6 years ago

By going against the convention and placing your proto files in a custom root you are making your life harder for no good reason.

you should always try to stick to maven conventions and avoid unnecessary customisations

Surely sticking to maven conventions would mean using the resource folder for proto files, rather than adding a proto folder to main? Proto files are not source code, but are resources used to transpile to source code.

Proto files are NOT a java file. They are used to generate source files - eg, java, c++, php etc. We use a custom folder as we us git submodules - we clone a git repo of proto files into the proto directory. These proto files are also used in other projects. I'm sure this is not that usual. Having an inflexible system that doesnt cater to using custom folders is what is going against convention.

Again, you are making your life more difficult by introducing packages,

As mentioned above, protos are essentially contracts - meaning they are not always up for change.

Before opening bugs against maven plugin, please make sure your proto files can be compiled by protoc on the command line.

Yes the files can be compiled using cli.

Anyway, I was able to solve this issue using https://github.com/os72/protoc-jar-maven-plugin

sergei-ivanov commented 6 years ago

Sources vs. resources and compiling vs. transpiling is a philosophical and/or linguistic debate and I have little desire to go there. Suffice to say that: a) quite a few code generating maven plugins use src/main/xxx by default for their inputs; b) this plugin is over 5 years old, and some decisions were taken long time ago and by other people, and I don't want to change the existing conventions unnecessarily for no good reason and break backwards compatibility for the existing users.

Also, I had customers/users with non-trivial configurations, including git submodules, packages and includes/excludes. There is enough flexibility in this maven plugin to achieve the desired effect. As it is generally the case with Maven, there's a certain degree of flexibility, but it is often intentionally limited, so that the users are discouraged from going radically against the conventions and creating monstrous build configurations. If one finds themselves fighting Maven, then they are either doing it wrong and in an un-Maven way, or they need a more suitable tool for the job, perhaps Gradle.

I am glad you managed to solve your problem with an alternative tool, which fits into the "use the most approprate tool for the job" philosophy. Good luck with your project!