search

bazelbuild / migration-tooling

Migration tools for Bazel
Apache License 2.0
44 stars 30 forks source link

About the time to build the transitive_maven_jar #51

Open GinFungYJF opened 6 years ago

GinFungYJF commented 6 years ago

Hi, I have tried the transitive_maven_jar. I find that the cost of re-fetching dependencies everytime is very heavy. It cost a lot of time. Are there any solution to speed up it?

petroseskinder commented 6 years ago

There are a few ideas, but it is still very much a work in progress at the moment.

To help us improve the tool, can you explain your general workflow? How many direct external dependencies are you listing? How frequently are you updating your external dependencies?

GinFungYJF commented 6 years ago

Thanks for the reply. Here is my workflow: Workflow 1: The initial setup. There are several modules in my project. I read the dependencies written in every pom.xml, and then I list these direct external dependencies in the transitive_maven_jar in WORKSPACE. I'm listing totally 41 direct external dependencies in the transitive_maven_jar. Finally I run bazel build @dependencies//... and it takes 634s to generate the generate_workspace.bzl.

Wrote /root/.cache/bazel/_bazel_root/e692933bf98f7caab6327d11bacc1d61/external/dependencies/generate_workspace.bzl
INFO: Found 0 targets...
INFO: Elapsed time: 634.882s, Critical Path: 0.01s

Workflow 2: Add a new external dependency. After I add a new external dependency in the transitive_maven_jar(for example, totally 42 direct external dependencies), I run bazel build @dependencies//... again. And it takes 569s to update the generate_workspace.bzl. 

Wrote /root/.cache/bazel/_bazel_root/e692933bf98f7caab6327d11bacc1d61/external/dependencies/generate_workspace.bzl
INFO: Found 0 targets...
INFO: Elapsed time: 569.320s, Critical Path: 0.27s

Whenever I add a new external dependency, it takes much time to update the generate_workspace.bzl .

petroseskinder commented 6 years ago

How frequently are you adding new dependencies? The general assumption we make is that you will not be frequently updating your external dependencies, so this performance cost is paid rarely.

The performance is very much dependent on your specific dependencies. If they have very deep transitive dependencies, then it will tend to consume a considerable amount of time.

GinFungYJF commented 6 years ago

I am not very often adding new dependencies, but the sha1 of some external dependencies changes frequently.