Gradle attempts to parse a JAR file as a POM file

[Never-D]

Background: when using Jenkins k8s plug-in for container construction, PVC is used for mounting Gradle for caching.

Problem: errors will be reported during concurrent construction. There are many kinds of specific error messages. Take the following two as examples:

1. xxx/. gradle/caches/jars-9/e8ba0b9cfd956eb11460c4be846359bc' to cache file-access. bin

2. [Fatal Error] xxxxx. jar:2:1: Content is not allowed in prolog.

Could not parse POM xxxx/. gradle/caches/modules-2/files-2.1/xxxxxx/xxxx/xxxx/9befde581e5649d76d5f89e165fa4ef94715b430/xxxxx. jar. Content is not allowed in prolog.

Expected solution: provide a way to support concurrent construction of gradle and share cache.

See the following for the specific construction error information:

Looking forward to the answer！！！

[Never-D]

Add a typical question: Background: it is consistent with the above problem scenario (occurs when the shared cache is built concurrently). Problem: the gradle file lock is occupied by other processes and cannot be killed. Expected solution: be able to build a solution to the problem. See the following figure for details:

[h0tk3y]

@Never-D Could you please elaborate on what exactly you are trying to achieve concurrently? Note that Gradle offers limited support for some of the use cases that assume reusing the Gradle home (including the caches) in concurrent builds.

[Never-D]

@h0tk3y Due to the large amount of cache, the official method of copying the cache to the gradle user home directory during construction is not feasible (the speed of copying the cache is too slow). Originally, this problem did not occur during concurrent construction in the virtual machine. I would like to ask whether the official can support such a scenario of concurrent construction: multiple construction processes read and write a cache at the same time, It can block (one is normal, the others block the sequence and wait) but will not throw an exception.

[Never-D]

@h0tk3y For the above several construction problems, does the official have any plans to repair or solve them? Expect to give a solution.

[abstratt]

Could you please provide the actual text output (instead of images)? Being able to see the original text of entire stacktrace (including nested exceptions) for the two errors mentioned will allow us to have a better idea of what may be going on.

Also, can you confirm which version of Gradle are you using? (it seems to be 6.8.3)

Finally, is the problem reproducible after you completely delete your version-specific cache location? (by default: $USER_HOME/.gradle/caches/6.8.3).

See user guide for more information on the directories Gradle uses.

[Never-D]

@abstratt gradle官方反馈问题.txt

[Never-D]

@abstratt The above file is the specific error log, and the error versions are 6.8.3 and 7.3.3

[Never-D]

@abstratt I encountered this problem again today gradle官方问题反馈2.txt

[abstratt]

Thanks for those two build outputs, that is much easier to make sense of.

Re: timeouts, this is a duplicate of https://github.com/gradle/gradle/issues/851 - there are three suggestions for that:

• using private Gradle homes for each container (no sharing, so no contention)
• copying the dependency cache into each container
• sharing a read-only dependency cache between multiple containers

It seems you said the second option above does not work for you. That would leave you with the other two options.

Your first attachment shows a different issue, one I still have to wrap my head around:

Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve xxxxxx.xxxx:cipher-context:1.2.20.rc5-RELEASE.
Caused by: org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.MetaDataParseException: Could not parse POM /root/gradle/xxxx_xxxxxxxxxxxxmobilesolution_1803/.gradle/caches/modules-2/files-2.1/xxxxxx.mobile/basic-constant/develop-xxxxxxxxxxxx-8.3.8-01-SNAPSHOT/9befde581e5649d76d5f89e165fa4ef94715b430/basic-constant-develop-xxxxxxxxxxxx-8.3.8-01-SNAPSHOT.jar
...
Caused by: org.xml.sax.SAXParseException; systemId: file:/root/gradle/xxxx_xxxxxxxxxxxxmobilesolution_1803/.gradle/caches/modules-2/files-2.1/xxxxxx.mobile/basic-constant/develop-xxxxxxxxxxxx-8.3.8-01-SNAPSHOT/9befde581e5649d76d5f89e165fa4ef94715b430/basic-constant-develop-xxxxxxxxxxxx-8.3.8-01-SNAPSHOT.jar; lineNumber: 2; columnNumber: 1; Content is not allowed in prolog.

It seems to be trying to parse XML from a JAR file location. I have no explanation for why that would ever happen. May be due to a corrupt cache. Does it happen after you clear the Gradle home dir?

[Never-D]

@abstratt Thank you for your answer!

First of all, answer your question: after the cache is clear, this situation disappears, and the construction can proceed normally. So I doubt whether there are some bugs in the shared cache of gradle. I hope to get some answers.

Secondly, for the three suggestions given, the second one is that we think the copy time is too long after trying, so it is not feasible; For the first proposal, because Jenkins' k8s containerized construction is used, the container will be automatically destroyed at the end of each construction (otherwise it will occupy too many machine resources), which is not feasible; For the third proposal, I want to make a specific confirmation: according to what you mean, all directories share a read-only cache a, then write to another cache B, and synchronize the contents of B to a at the end of the construction?

Finally, there is another problem. Recently, we found a large number of concurrent builds that there are few errors (almost no) that file locks cannot be released during the concurrent build of shared cache in the old version of gradle (4.x, 5.x, etc.). Is it a bug in the new version after 6.8.3 (especially 7.3.3 and above required by jdk17)?

[abstratt]

For the third proposal, I want to make a specific confirmation: according to what you mean, all directories share a read-only cache a, then write to another cache B, and synchronize the contents of B to a at the end of the construction?

There is no mechanism for automatically updating the read-only cache. To keep it as up-to-date as possible, you could use one or more seed builds to prime it (say, every night). From Sharing the dependency cache with other Gradle instances:

In a CI environment, it’s a good idea to have one build which "seeds" a Gradle dependency cache, which is then copied to a different directory. This directory can then be used as the read-only cache for other builds. You shouldn’t use an existing Gradle installation cache as the read-only cache, because this directory may contain locks and may be modified by the seeding build.

[abstratt]

Finally, there is another problem. Recently, we found a large number of concurrent builds that there are few errors (almost no) that file locks cannot be released during the concurrent build of shared cache in the old version of gradle (4.x, 5.x, etc.). Is it a bug in the new version after 6.8.3 (especially 7.3.3 and above required by jdk17)?

I am not sure. Could you please search for existing issues using some relevant keywords from the error, and if you cannot find existing issues, report as a new issue (including details)? Ideally, each GH issue should cover a single problem.

[abstratt]

Given the problem with a JAR url being processed as a POM cleared up after caches were deleted, this looks like a corrupt cache issue.