Closed svilenvul closed 4 months ago
note from my investigation:
ICodeBase
,IClassPath
is AutoCloseable
.IClassPath
is responsible to close all ICodeBase
instances inside itFindBugs2#clearCaches()
closes generated IClassPath
FindBugs2#clearCaches()
for nowSo I guess that we should invoke this FindBugs2#clearCaches()
method, or create classloader to GC FindBugs2.classpath
that is singleton field?
I will test following patch later.
diff --git a/spotbugs/src/main/java/edu/umd/cs/findbugs/FindBugs2.java b/spotbugs/src/main/java/edu/umd/cs/findbugs/FindBugs2.java
index dc061d2a8..1be3e9691 100644
--- a/spotbugs/src/main/java/edu/umd/cs/findbugs/FindBugs2.java
+++ b/spotbugs/src/main/java/edu/umd/cs/findbugs/FindBugs2.java
@@ -1185,7 +1185,7 @@ public class FindBugs2 implements IFindBugsEngine {
FindBugs.runMain(findBugs, commandLine);
-
+ findBugs.clearCaches();
}
findBugs.clearCaches()
are already invoked in FindBugs2#execute()
which is called by FindBugs#runMain(...)
.https://github.com/spotbugs/spotbugs/issues/589 is similar with this issue, but this issue's target is mainly jar files in classpath to scan.
@svilenvul let me confirm: are your Maven uses -T
option, to run modules in parallel?
@svilenvul let me confirm: are your Maven uses -T option, to run modules in parallel?
No, currently it is not being used.
We started investigating the problem and found out that there is a huge number of opened files (spotbugs plugins).
Almost all opened file isn't not spotbugs plugin, but libraries in aux classpath.
In current architecture, spotbugs keeps all jar files opened during analysis (because IClassPath
is keep opened), so we should face problem like this. To fix this issue, I think we need to change internal architecture to handle libraries in aux & app classpaths as stream.
Has there been any progress? We are now seeing this frequently on our Jenkins CI server (we are using -T and there can also be concurrent builds).
This issue is likely related to #535 that has existed in this code base since its original inception with findbugs. Has to do with the -T and it getting locks as system.in was being called. This is to be removed in upcoming release 4.7.3.1 within the week.
closing as believe fixed with 4.7.3.1. If still an issue, please open a new ticket based off current data.
We are running the spotbugs-maven-plugin programmatically in a multi-module Maven project. This is why it is difficult to describe the exact spotbugs maven plugin configuration, this is why I will attach part of the Maven debug logs, where this can be seen:
spotbugs-config.txt
While trying to set up a Jenkins job, we faced ‘Too many open files’ problem causing the build to fail.
We started investigating the problem and found out that there is a huge number of opened files (spotbugs plugins).
I attach here the output of the following command
ls -l /proc/MAVEN_PID/fd/ > openfiles.txt
:openfiles.txt
They are all located in the target directory.
Used spotbugs-maven-plugin/3.1.3 with spotbugs 3.1.2