search

whitesource / unified-agent-distribution

51 stars 48 forks source link

Whitesource agent jar not scanning specified dir in jenkins + throwing Exception thread "main" java.lang.OutOfMemoryError: GC overhead limit exceed #27

Closed nirupbbnk closed 2 years ago

nirupbbnk commented 2 years ago

whitesource agent jar is scanning directory specified along with it ,it is scanning another directory in jenkins.

[INFO] [2022-01-22 01:39:36,332 +0000] - Scanning directories [/var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS, /tmp/ws-ua_20220122010754_IPJCFV/archiveExtraction_XHMMNY/RZMXZB/20220122010756] for matching source/binary file types (may take a few minutes) [INFO] [2022-01-22 01:53:11,287 +0000] - Total files found according to the includes/excludes pattern: 820,809

java -jar $WSS/wss-unified-agent.jar -c $WSS/config.properties -d /var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS

It should scan only files present in /var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS, but it is taking files present in /tmp/ws-ua_20220122010754_IPJCFV/archiveExtraction_XHMMNY/RZMXZB/20220122010756.

In path specified we dont have 8,20,000 , I dont know from where 8,20,000 files are coming.

Also at the end we are getting below error

[INFO] [2022-01-22 02:25:49,888 +0000] -

-------------------- End: Scan Files Matching Includes Pattern --------- Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Arrays.java:3332) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448) at java.lang.StringBuilder.append(StringBuilder.java:136) at ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:39) at ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:115) at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:165) at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39) at ch.qos.logback.core.encoder.LayoutWrappingEncoder.encode(LayoutWrappingEncoder.java:116) at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:230) at ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:235) at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:102) at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84) at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51) at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270) at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257) at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421) at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383) at ch.qos.logback.classic.Logger.debug(Logger.java:494) at org.whitesource.utils.logger.LoggerFS.debug(LoggerFS.java:118) at org.whitesource.agent.FileSystemScanner.createProjects(FileSystemScanner.java:537) at org.whitesource.fs.scanOrigins.GeneralScanOrigin.getProjects(GeneralScanOrigin.java:189) at org.whitesource.fs.scanOrigins.GeneralScanOrigin.scan(GeneralScanOrigin.java:98) at org.whitesource.fs.scanOrigins.ScanOrigin.runOriginScan(ScanOrigin.java:36) at org.whitesource.fs.FileSystemAgent.createProjects(FileSystemAgent.java:174) at org.whitesource.fs.Main.scanProjects(Main.java:115) at org.whitesource.fs.Main.main(Main.java:90)

LenaKleyner commented 2 years ago

Hi @nirupbbnk

We will need more information to evaluate your issue. Can you please open a support ticket with all the relevant information (the Unified Agent execution command, its settings, and the output log)?

Thanks!

Lena

nirupbbnk commented 2 years ago

HI @LenaKleyner ,

WSS agent version : 21.12.2 Comand executed: java -jar $WSS/wss-unified-agent.jar -c $WSS/config.properties -d /var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS

Thanks

LenaKleyner commented 2 years ago

Thanks @nirupbbnk but we really recommend opening a support ticket via support@whitesourcesoftware.com with all the relevant information (including the configuration).

Thanks!

nabeelsaabna commented 2 years ago

@nirupbbnk the additional directory being scanned: /tmp/ws-ua_20220122010754_IPJCFV/archiveExtraction_XHMMNY/RZMXZB/20220122010756 is related to the archive extraction, for more details please check: https://whitesource.atlassian.net/wiki/spaces/WD/pages/1544880156/Unified+Agent+Configuration+Parameters#Archive-Properties

nabeelsaabna commented 2 years ago

closing the issue as this is the expected behavior according to the provided unified-agent configurations