Transitive dependencies are not always included

[arjantop]

Description

When using generate_workspace transitive dependencies are not always included in the exports parameter of java_library. This does not always happen (one guess would be that this happens when two artifacts share some common dependencies).

Example

bazel run //src/tools/generate_workspace -- --artifact=com.twitter:twitter-server_2.11:1.25.0

Output:

java_library(
    name = "com_twitter_twitter_server_2_11",
    visibility = ["//visibility:public"],
    exports = [
        "@com_twitter_twitter_server_2_11//jar",
        "@com_fasterxml_jackson_core_jackson_annotations//jar",
        "@com_fasterxml_jackson_core_jackson_core//jar",
        "@com_fasterxml_jackson_core_jackson_databind//jar",
        "@com_fasterxml_jackson_module_jackson_module_paranamer//jar",
        "@com_fasterxml_jackson_module_jackson_module_scala_2_11//jar",
        "@com_github_ben_manes_caffeine_caffeine//jar",
        "@com_google_code_findbugs_jsr305//jar",
        "@com_google_guava_guava//jar",
        "@com_thoughtworks_paranamer_paranamer//jar",
        "@com_twitter_finagle_base_http_2_11//jar",
        "@com_twitter_finagle_core_2_11//jar",
        "@com_twitter_finagle_http_2_11//jar",
        "@com_twitter_finagle_netty4_2_11//jar",
        "@com_twitter_finagle_netty4_http_2_11//jar",
        "@com_twitter_finagle_thrift_2_11//jar",
        "@com_twitter_finagle_toggle_2_11//jar",
        "@com_twitter_finagle_zipkin_core_2_11//jar",
        "@com_twitter_libthrift//jar",
        "@com_twitter_scrooge_core_2_11//jar",
        "@com_twitter_util_app_2_11//jar",
        "@com_twitter_util_cache_2_11//jar",
        "@com_twitter_util_codec_2_11//jar",
        "@com_twitter_util_collection_2_11//jar",
        "@com_twitter_util_core_2_11//jar",
        "@com_twitter_util_events_2_11//jar",
        "@com_twitter_util_function_2_11//jar",
        "@com_twitter_util_hashing_2_11//jar",
        "@com_twitter_util_jvm_2_11//jar",
        "@com_twitter_util_lint_2_11//jar",
        "@com_twitter_util_logging_2_11//jar",
        "@com_twitter_util_registry_2_11//jar",
        "@com_twitter_util_stats_2_11//jar",
        "@commons_lang_commons_lang//jar",
        "@io_netty_netty//jar",
        "@io_netty_netty_buffer//jar",
        "@io_netty_netty_codec//jar",
        "@io_netty_netty_codec_http//jar",
        "@io_netty_netty_codec_socks//jar",
        "@io_netty_netty_common//jar",
        "@io_netty_netty_handler//jar",
        "@io_netty_netty_handler_proxy//jar",
        "@io_netty_netty_resolver//jar",
        "@io_netty_netty_transport//jar",
        "@javax_servlet_servlet_api//jar",
        "@org_scala_lang_modules_scala_parser_combinators_2_11//jar",
        "@org_scala_lang_scala_library//jar",
        "@org_scala_lang_scala_reflect//jar",
        "@org_slf4j_slf4j_api//jar",
    ],
)

bazel run //src/tools/generate_workspace -- \
  --artifact=com.twitter:finagle-thrift_2.11:6.40.0 \
  --artifact=com.twitter:twitter-server_2.11:1.25.0

Output (transitive dependencies are missing):

java_library(
    name = "com_twitter_twitter_server_2_11",
    visibility = ["//visibility:public"],
    exports = [
        "@com_twitter_twitter_server_2_11//jar",
        "@com_fasterxml_jackson_core_jackson_core//jar",
        "@com_fasterxml_jackson_core_jackson_databind//jar",
        "@com_fasterxml_jackson_module_jackson_module_scala_2_11//jar",
        "@com_google_guava_guava//jar",
        "@com_twitter_finagle_base_http_2_11//jar",
        "@com_twitter_finagle_core_2_11//jar",
        "@com_twitter_finagle_http_2_11//jar",
        "@com_twitter_finagle_netty4_2_11//jar",
        "@com_twitter_finagle_netty4_http_2_11//jar",
        "@com_twitter_finagle_thrift_2_11//jar",
        "@com_twitter_finagle_toggle_2_11//jar",
        "@com_twitter_finagle_zipkin_core_2_11//jar",
        "@com_twitter_libthrift//jar",
        "@com_twitter_scrooge_core_2_11//jar",
        "@com_twitter_util_app_2_11//jar",
        "@com_twitter_util_codec_2_11//jar",
        "@com_twitter_util_collection_2_11//jar",
        "@com_twitter_util_core_2_11//jar",
        "@com_twitter_util_events_2_11//jar",
        "@com_twitter_util_jvm_2_11//jar",
        "@com_twitter_util_lint_2_11//jar",
        "@com_twitter_util_logging_2_11//jar",
        "@com_twitter_util_registry_2_11//jar",
        "@com_twitter_util_stats_2_11//jar",
        "@commons_lang_commons_lang//jar",
        "@io_netty_netty_codec_http//jar",
        "@org_scala_lang_scala_library//jar",
    ],
)

[arjantop]

I think the problem is here: https://github.com/bazelbuild/bazel/blob/master/src/tools/generate_workspace/src/main/java/com/google/devtools/build/workspace/maven/Resolver.java#L166

New instance of Rule is created and transitive dependencies are resolved and added only in the case that the dependency is new. If the dependency already exists the dependencies will never be populated and https://github.com/bazelbuild/bazel/blob/master/src/tools/generate_workspace/src/main/java/com/google/devtools/build/workspace/maven/Resolver.java#L185 will have no effect.

[kant111]

any progress here?

[dslomov]

generate-workspace is no longer supported.