search

bazel-contrib / rules_jvm_external

Bazel rules to resolve, fetch and export Maven artifacts
Apache License 2.0
333 stars 255 forks source link

Compile Line Length Limit hit under Windows 10.0 with bazel 4.0.0 #623

Open lee-csu opened 2 years ago

lee-csu commented 2 years ago

Description of the problem / feature request:

I tried to compile the Java project on Windows 10, but because the project has too many dependencies, an error message is displayed during dependency parsing, indicating that command is longer than CreateProcessW's limit. My maven_install() defines nearly 500 dependencies.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

workspace maven_install( artifacts = [ "org.apache.zookeeper:zookeeper-jute:3.6.0", "org.apache.zookeeper:zookeeper-jute:3.6.2", "org.apache.zookeeper:zookeeper:3.6.2-h0.gdd.sop.r18", "io.grpc:grpc-api:1.36.1", ...

....nearly 500 dependencies

] )

What operating system are you running Bazel on?

windows10 Bazel verison：4.0.0 rules_jvm_external version：4.0

What's the output of bazel info release?

no such package '@maven//': Error while fetching artifact with coursier: java.io.IOException: ERROR: src/main/native/windows/process.cc(165): CreateProcessWithExplicitHandles("D:\tools\javajdk\bin\java" -noverify -jar C:/users/xxx/esyrauaw/external/maven/coursier fetch org.apache.zookeeper:zookeeper-jute:3.6.0 org.apache.zookeeper:zookeeper-jute:3.6.2 org.apache.zookeeper:zookeeper:3.6.2-h0.gdd.sop.r18 io.grpc:grpc-api:1.36.1 io.grpc:grpc-context:1.36.1 io.grpc:grpc-core:1.36.1 io.grpc:grpc-grpclb:1.36.1 io.grpc:grpc-netty:1.36.1 io.grpc:grpc-protobuf- lite:1.36.1 io.grpc:grpc-protobuf:1.36.1 io.grpc:grpc-stub:1.36.1 org.quartz-scheduler.in(...)): command is longer than CreateProcessW's limit (32767 characters)

jin commented 2 years ago

This is a limitation of coursier -- I don't think coursier supports argument files/flag files, but that would be the approach to fix this problem.

cjohnstoniv commented 2 years ago

Has anyone opened this as an issue with Coursier yet?

I raised one: https://github.com/coursier/coursier/issues/2443

cjohnstoniv commented 1 year ago

Coursier seems to have added the ability now to specify a file (PR: https://github.com/coursier/coursier/pull/2604) to pass the arguments as a file.

Without personally ever touching the lower level Bazel code, would any maintainers have any pointers and/or a quick way we could make use of this with Bazel?

jin commented 1 year ago

https://github.com/bazelbuild/rules_jvm_external/blob/d211f3b057528f06d0414a56c10ddb8849ede78b/coursier.bzl#L656 is where the coursier command is constructed. One way is to pipe all of the artifacts into a file with repository_ctx.file and pass that file as an arg into the coursier command with the --dependency-file flag.

cjohnstoniv commented 1 year ago

Thank you @jin , will try to find some time later this week/this weekend to see I can get this working on my own custom JVM rules build.