bazelbuild / migration-tooling

Migration tools for Bazel
Apache License 2.0
45 stars 30 forks source link

Allow specifying exclusions to generate_workspace #24

Closed cpenner-va closed 7 years ago

cpenner-va commented 7 years ago

One issue my team is having with the existing generate_workspace tool (in the bazelbuild repo) is that we're generating dependencies for a maven artifact (google-cloud-logging) which includes some java netty dependencies, but in another place in the application we're including the pubfref protobuf rules for building java protobufs for grpc, and so are including netty deps from there via:

"@org_pubref_rules_protobuf//java:netty_runtime_deps"

So we're having an issue where we have two sets of netty dependencies being included of conflicting versions. Ideally I would like to tell generate_workspace that I have certain dependencies satisfied already, perhaps via a command line flag I can tell it that certain netty targets are satisfied externally through the pubref rules and it will generate a BUILD where the rules there depend on the pubref netty instead of duplicating the dependency.

I understand that we're using generate_workspace in an unintended way (we run it again each time that we add a new dependency, this is because the -b flag of the other tool is not working).

Is there a recommended solution for overlapping dependencies like this?

Thanks.

kchodorow commented 7 years ago

It sounds like you probably want to be able to say, "when you see netty protobuf stuff, use "@org_pubref_rules_protobuf//java:netty_runtime_deps" instead," rather than exclude them entirely. E.g., if you do --artifact=x:y:1.2.3 and that depends on protobuf rules, you still would want "@org_pubref_rules_protobuf//java:netty_runtime_deps" in its deps. Is that correct?

cpenner-va commented 7 years ago

Yes that sounds like a good solution; The current issue I'm having is that we're generating deps for com.google.cloud:google-cloud-logging:1.1.0 via the -a flag, but our target (a grpc server) includes netty deps which overlap with the pubref rules (due to grpc). That is, our target includes BOTH of "com_google_cloud_google_cloud_logging" AND "@org_pubref_rules_protobuf//java:netty_runtime_deps", both of which are pulling in netty deps.

In WORKSPACE I tried using pubref's exclude to avoid pulling in some of its dependencies to allow it to at least build, but this seems to be resulting in problems at runtime, perhaps because the google cloud logging netty dependency versions are incompatible with pubref's versions of netty? Or maybe they're being included twice? I'm not certain. I did notice however that google-cloud-logging's netty dependencies are quite out of date.

cpenner-va commented 7 years ago

Here's the error we're dealing with right now; it looks like we're just missing an include of netty-codec, but that should be included by "@org_pubref_rules_protobuf//java:netty_runtime_deps"; so I'm a little confused

Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/handler/codec/Headers
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at io.grpc.netty.NettyServer.allocateSharedGroups(NettyServer.java:186)
        at io.grpc.netty.NettyServer.start(NettyServer.java:114)
        at io.grpc.internal.ServerImpl.start(ServerImpl.java:161)
        at io.grpc.internal.ServerImpl.start(ServerImpl.java:83)
        at com.vendasta.adwords.grpc.GrpcServer.start(GrpcServer.java:34)
        at com.vendasta.adwords.grpc.GrpcServer.main(GrpcServer.java:63)
Caused by: java.lang.ClassNotFoundException: io.netty.handler.codec.Headers
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 18 more