search

jvolkman / intellij-protobuf-editor

Protocol Buffers for IntelliJ-based IDEs
Apache License 2.0
122 stars 15 forks source link

Add libraries and sdk classes roots file resolve provider #29

Closed devkanro closed 3 years ago

devkanro commented 4 years ago

Resolve with SDK classes and libraries

This PR provided resolving file dependencies in SDK classes and libraries.

Checklist

You can merge this PR after all check is done.

Description

Sometimes we will use protos in three-party libraries, intellij-protobuf-editor can't resolve those dependencies right now.

We add two file resolve providers to do that:

Some screenshots

Snipaste_2020-07-21_11-55-35

Before @jvolkman refactored the FileResolveProvider and ProjectSettingsConfigurator, you can download the modified plugin and install it from disk.

protobuf-editor.zip

jvolkman commented 4 years ago

Thanks. I don't think I'll accept this as is since I'm planning to refactor the FileResolveProvider and ProjectSettingsConfigurator concepts, but I'll look into adding support for pulling protos from libraries.

Do you have any examples of libraries that include .proto files (in the .jar) other than protobuf-java? In my experience libraries only have the generated classes and maybe the generated source.

devkanro commented 4 years ago

In our business case, we use the customized proto compiler plugin, protobuf runtime and gRPC runtime ButterCam/sisyphus, we have many customized options, we need to provide them in jar for other component usages.

In the common case, google provided The Google API Improvement Proposals, if you want to follow the guide book, you must use some options and protos included in some common libraries like com.google.api.grpc:proto-google-common-protos

It will be many benefits to make protos distributable, you can use not only the generated classes in your java codes but also the proto messages in your proto.

kshchepanovskyi commented 4 years ago

In my company we are using libraries with proto files too (jar file contains compiled java classes and proto sources).

jvolkman commented 4 years ago

Thanks! Sounds like looking at libraries makes sense, then. Are the .proto files generally rooted at the top of the jar (i.e., not under a protos/ directory or something)?

devkanro commented 4 years ago

Yes, .proto files just are the same as other resources in the jar.

jmpicnic commented 4 years ago

Thanks for the reference @devkanro

As an additional note, the project we are working on is a multi-team project with multiple repositories and micro-services that export their .proto together with their Java/Kotlin grpc stubs (client and server separately) to an artifactory repo. Teams create and manage dependencies between micro services by deciding what libraries to use. For "pure" use in implementation, the .proto are not needed, but there are many cases where a message type (or more likely a sub-type) gets reused in multiple service endpoints written by different teams. In this case, we need access to the *.proto from the editor plug-in

jvolkman commented 4 years ago

For background, at Google where I originally developed this plugin, .proto files generally weren't included with build artifacts that were indexed by IntelliJ. Additionally at Google, all source code exists in one single giant repository that's magically available to developer workstations via a special userspace filesystem. And all .proto imports are relative to the root of this repo. So this plugin pretty much side-steps standard import resolution techniques in IntelliJ and instead simply looks for files relative to a set of defined import paths. When people use the plugin within Google, there's usually just one path defined which points to the user's current view of that giant repository.

Obviously this isn't sufficient for use cases outside of Google. We could configure the list of import paths to be equal to the list of library paths, which I believe is what this PR is attempting to do. But I think this approach is a bit too coarse since I assume most library jars in a project won't actually contain .proto files. And so completing an import statement will provide a number of directory suggestions that end up being dead ends.

What I think I'd like to do is create an index if proto import paths and use that as the primary form of resolution. The index will pull from libraries and source roots, presumably, and its use will allow more intelligent completion when typing imports. I'd also like to use an index to auto-import selected symbols, but that's only tangentially related.

efenglu commented 4 years ago

So just some context the previous version of this plugin did exactly what I need: https://github.com/protostuff/protobuf-jetbrains-plugin

It correctly added all jars to the path to search for protos.

We use maven extensively and shared libraries with protos. This worked great for our team of teams with dozens of git repos and multiple versions.

bjaglin commented 3 years ago

So just some context the previous version of this plugin did exactly what I need: https://github.com/protostuff/protobuf-jetbrains-plugin

Same situation here. Since that plugin is deprecated, it's no longer available in the market place and therefore we are seeing that "regression" for fresh IntelliJ installations using this one (which is great otherwise!).

Do you have any examples of libraries that include .proto files (in the .jar) other than protobuf-java? In my experience libraries only have the generated classes and maybe the generated source.

In our case, we depend on protos included in these JARs

devkanro commented 3 years ago

@bjaglin @efenglu

Before @jvolkman refactored the FileResolveProvider and ProjectSettingsConfigurator, you can download the modified plugin and install it from disk.

protobuf-editor.zip