search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.44k stars 1.28k forks source link

NVD API request failures #6149

Closed Casperxian closed 11 months ago

Casperxian commented 11 months ago

Describe the bug Fail to run DependencyCheckAnalyze, no status code thrown as I cancelled the build due to it being over 1 hour build time.

Version of dependency-check used The problem occurs using version 9.0.1 gradle plugin

Log file

Task :dependencyCheckAnalyze Verifying dependencies for project AccessCheck Checking for updates and analyzing dependencies for vulnerabilities

NVD API request failures are occurring; retrying request for the 5 time NVD API request failures are occurring; retrying request for the 6 time NVD API request failures are occurring; retrying request for the 7 time NVD API request failures are occurring; retrying request for the 8 time NVD API request failures are occurring; retrying request for the 9 time NVD API request failures are occurring; retrying request for the 10 time NVD API request failures are occurring; retrying request for the 11 time NVD API request failures are occurring; retrying request for the 5 time NVD API request failures are occurring; retrying request for the 6 time NVD API request failures are occurring; retrying request for the 7 time NVD API request failures are occurring; retrying request for the 8 time NVD API request failures are occurring; retrying request for the 9 time NVD API request failures are occurring; retrying request for the 10 time NVD API request failures are occurring; retrying request for the 11 time NVD API request failures are occurring; retrying request for the 5 time NVD API request failures are occurring; retrying request for the 6 time NVD API request failures are occurring; retrying request for the 7 time NVD API request failures are occurring; retrying request for the 8 time NVD API request failures are occurring; retrying request for the 9 time NVD API request failures are occurring; retrying request for the 10 time NVD API request failures are occurring; retrying request for the 11 time

Error updating the NVD Data org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:338) at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:110) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:904) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:709) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:635) at org.owasp.dependencycheck.gradle.tasks.AbstractAnalyze.analyze(AbstractAnalyze.groovy:100) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29) at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:248) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68) at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:233) at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:216) at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:199) at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:166) at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:105) at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:44) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:59) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:56) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:56) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:44) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:67) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:37) at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41) at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74) at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:50) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:28) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.executeDelegateBroadcastingChanges(CaptureStateAfterExecutionStep.java:100) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:72) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:50) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:40) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:29) at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:179) at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:70) at org.gradle.internal.Either$Right.fold(Either.java:175) at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:68) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:46) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22) at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:91) at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:55) at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:37) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:65) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:36) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:77) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:38) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:94) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:49) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:71) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:45) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNonEmptySources(SkipEmptyWorkStep.java:177) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:81) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:53) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:36) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:23) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:75) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:41) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.lambda$execute$2(ExecuteWorkBuildOperationFiringStep.java:66) at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:66) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:38) at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:32) at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:293) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:21) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:47) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:34) at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:64) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:146) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:135) at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52) at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:42) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:331) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:318) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.lambda$execute$0(DefaultTaskExecutionGraph.java:314) at org.gradle.internal.operations.CurrentBuildOperationRef.with(CurrentBuildOperationRef.java:80) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:314) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:303) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:463) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:380) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:47) at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base@17.0.7/java.lang.Thread.run(Thread.java:833) Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: java.lang.InterruptedException: sleep interrupted at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:351) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:356) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:356) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:356) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:356) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:356) at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:317) ... 133 more Caused by: java.lang.InterruptedException: sleep interrupted at java.base/java.lang.Thread.sleep(Native Method) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.getCompletedFuture(NvdCveClient.java:396) at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:315) ... 139 more Failed to initialize the RetireJS repo org.owasp.dependencycheck.data.update.exception.UpdateException: Failed to initialize the RetireJS repo at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:141) at org.owasp.dependencycheck.data.update.RetireJSDataSource.update(RetireJSDataSource.java:89) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:904) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:709) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:635) at org.owasp.dependencycheck.gradle.tasks.AbstractAnalyze.analyze(AbstractAnalyze.groovy:100) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29) at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:248) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68) at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:233) at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:216) at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:199) at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:166) at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:105) at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:44) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:59) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:56) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:56) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:44) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:67) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:37) at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41) at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74) at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:50) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:28) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.executeDelegateBroadcastingChanges(CaptureStateAfterExecutionStep.java:100) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:72) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:50) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:40) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:29) at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:179) at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:70) at org.gradle.internal.Either$Right.fold(Either.java:175) at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:68) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:46) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22) at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:91) at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:55) at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:37) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:65) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:36) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:77) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:38) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:94) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:49) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:71) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:45) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNonEmptySources(SkipEmptyWorkStep.java:177) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:81) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:53) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:36) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:23) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:75) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:41) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.lambda$execute$2(ExecuteWorkBuildOperationFiringStep.java:66) at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:66) at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:38) at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:32) at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:293) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:21) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:47) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:34) at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:64) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:146) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:135) at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52) at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:42) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:331) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:318) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.lambda$execute$0(DefaultTaskExecutionGraph.java:314) at org.gradle.internal.operations.CurrentBuildOperationRef.with(CurrentBuildOperationRef.java:80) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:314) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:303) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:463) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:380) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:47) at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base@17.0.7/java.lang.Thread.run(Thread.java:833) Caused by: org.owasp.dependencycheck.exception.WriteLockException: Unable to obtain the update lock, skipping the database update. Skipping the database update. at org.owasp.dependencycheck.utils.WriteLock.lock(WriteLock.java:199) at org.owasp.dependencycheck.utils.WriteLock.(WriteLock.java:134) at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:136) ... 133 more Unable to continue dependency-check analysis.

To Reproduce Steps to reproduce the behavior: Run: "./gradlew dependencyCheckAnalyze"

Expected behavior CVE Report generated

Additional context I have tried disabling my proxy and also commenting out jdk.http.auth.tunneling.disabledSchemes=Basic in java .net.properties but so far it does not work for me, i do get status code 407 when I enabled both proxy and uncomment dk.http.auth.tunneling.disabledSchemes but also the get the same issue as above sometimes.

Sjoerd97 commented 11 months ago

Can you share your the gradle configuration where you configure the dependencyCheck plugin?

Casperxian commented 11 months ago

here it is

extensions.findByType(org.owasp.dependencycheck.gradle.extension.DependencyCheckExtension::class.java)?.apply { format = "JSON" failOnError = false nvd.apiKey = "########-####-####-####-########### nvd.validForHours = 24 }

ccerrillo commented 11 months ago

I can confirm the same issue using Maven. And this is also related #6147

NathanDotTo commented 11 months ago

Same issue here. My build for a project called "appsensor" is encapsulated in Docker files here: https://github.com/NathanDotTo/appsensor

jeremylong commented 11 months ago

are you behind a proxy?

NathanDotTo commented 11 months ago

No. On a private home network.On 27 Nov 2023, at 13:12, Jeremy Long @.***> wrote: are you behind a proxy?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

jeremylong commented 11 months ago

great... the the 407 is actually coming from the NVD somehow? That is going to cause some confusion for those that are actually behind a proxy...

JasonShuyinta commented 11 months ago

I'm having the same issue I guess, these are some of the logs if it could help: I'm using home network as well, so I'm not behind a proxy.

Task :dependencyCheckAnalyze Error updating the NVD Data org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:338) at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:110) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:904) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:709) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:635)
[....] Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: NVD Returned Status Code: 504 at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:346) at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:317) ... 127 more

NathanDotTo commented 11 months ago

I suspect that the dependency data update/download has to be decoupled from the code build and test. Perhaps a different stage for the data download?On 27 Nov 2023, at 13:21, Jeremy Long @.***> wrote: great... the the 407 is actually coming from the NVD somehow? That is going to cause some confusion for those that are actually behind a proxy...

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

jeremylong commented 11 months ago

@JasonShuyinta that is a different status code. The 504 should be handled in the next release.

echalone commented 11 months ago

We had the 504 error today but then sometimes that works and then we get endless "NVD API request failures are occurring; retrying request for the x time" error as well now

codecutr commented 11 months ago

@jeremylong Is the 504 issue still unresolved? I am experiencing the same error as @JasonShuyinta.

thisjustin816 commented 11 months ago

I tried adding an api key in azure pipelines to get around the 504 issue and now just get 404s

Casperxian commented 11 months ago

Even without proxy it will not complete a run even after waiting an hour. curling to NVD NIST works for me on terminal so Im not sure what is the issue for the build to not be completed with many retries.

The link of the curl: https://services.nvd.nist.gov/rest/json/cvehistory/2.0?cveId=CVE-2019-1010218

ccerrillo commented 11 months ago

NVD has put a temporary block https://github.com/jeremylong/DependencyCheck/issues/6107#issuecomment-1828709818

mucst commented 11 months ago

Also got 403 with plugin version 9.0.1. Earlier with 9.0.0 I got "database is closed", 503 error.

config:

dependencyCheck {

    // exclude the following configurations as they are not relevant for the resulting artifact
    scanConfigurations = ['runtime', 'runtimeClasspath', 'runtimeElements', 'runtimeOnly']

    analyzers {
        assemblyEnabled = false
    }

    failOnError = false
    format = isCiServer ? 'XML': 'HTML'
    outputDirectory = file("$buildDir/reports/dependency-check")

    hintsFile = dependencyCheckHintsFile
    suppressionFiles += dependencyCheckSuppressionFile
}

Downgrading to latest of 8.x (8.4.3 today) fixed it.

Sjoerd97 commented 11 months ago

Down grading should never be the fix. Beside 8.4.3 will stop working.

echalone commented 11 months ago

Yeah well the problem is 9.x never started working for many of us. We're still using 8.4.3 as this works

pnsag commented 11 months ago

Hopefully we can overcome that situation very soon, as this is blocking.

stephan-uhlmann commented 11 months ago

Can confirm that with version 9.0.1 and 9.0.0 download of the NVD database is not working, while with 8.4.3 it still is working. (//edit: typo, 9.x is not working.) @Sjoerd97 What's the reason why 8.4.3 will stop working?

norrisjeremy commented 11 months ago

Can confirm that with version 9.0.1 and 9.0.0 download of the NVD database is working, while with 8.4.3 it still is working. @Sjoerd97 What's the reason why 8.4.3 will stop working?

Maybe try reading the notice on the main page?

thisjustin816 commented 11 months ago

Can confirm that with version 9.0.1 and 9.0.0 download of the NVD database is working, while with 8.4.3 it still is working.

@Sjoerd97 What's the reason why 8.4.3 will stop working?

Did you mean not working? I'm still getting a 403.

pnsag commented 11 months ago

Can confirm that with version 9.0.1 and 9.0.0 download of the NVD database is working

How did you managed that permanently. I had it working at the weekend only, than stopped again.

Sjoerd97 commented 11 months ago

Big chance it worked in the weekend because corporates were not spamming the endpoint. I hope nvd will scale to the required capacity before deprecated the previous data infeeds

Sjoerd97 commented 11 months ago

Beside, I get that 9.+ Never worked but suggesting 8.4.3 as a solution to somebody that experiences a bug when upgrading isnt really constructive

michalszelagsonos commented 11 months ago

Note, for folks that use node's owasp-dependency-check package, there is no way to downgrade since the package pulls the latest available release. I opened a feature request to provide that override but that won't help those projects out of this jam right now.

OneAndOnlySeabass commented 11 months ago

I am getting 403 errors as well even though I'm using an official NVD API key.

gbrinkmann commented 11 months ago

Experiencing 403 too while testing 9.0.1 without api key before migrating to 9.0.1 in our git pom.

manojKumar-L commented 11 months ago

@jeremylong I am also getting 403 errors in the latest 9.0.1 version and even though using an official NVD API key in my Android project, provided the log SS below for reference image image

sae-artem commented 11 months ago

We also get 403 errors in the latest 9.0.1 version (not using NVD API key though)

Spanish93 commented 11 months ago

Ok stop spamming with all your 403 errors, problem it's same for all and already known.

NVD has put a lock see comment here : https://github.com/jeremylong/DependencyCheck/issues/6107#issuecomment-1828709818

aikebah commented 11 months ago

Closing as a duplicate of 6107

aikebah commented 11 months ago

reopened.. to have an item open as 6107 was closed

ccerrillo commented 11 months ago

@jeremylong Any news from the API maintainers?

echalone commented 11 months ago

Can confirm that with version 9.0.1 and 9.0.0 download of the NVD database is working, while with 8.4.3 it still is working. @Sjoerd97 What's the reason why 8.4.3 will stop working?

Maybe try reading the notice on the main page?

Yeah well it would be a bad idea to stop supporting 8.4.3 if 9.x isn't working for anybody because the NVD database is blocking the way Dependency Check 9.x is accessing the NVD database ^^ See notice by developer here: https://github.com/jeremylong/DependencyCheck/issues/6107#issuecomment-1828709818

It's not like us user can do anything here to get it working...

jeremylong commented 11 months ago

Update - 11/30/2023

Apparently the virtualMatchString filter of cpe:2.3:a was causing performance issues. I'm still in conversations - but plan on releasing the next version of dependency-check tomorrow morning. Hopefully, the release will resolve most issues although we may still have an issue with proxies that require authentication.

echalone commented 11 months ago

get it working...

Update - 11/30/2023

Apparently the virtualMatchString filter of cpe:2.3:a was causing performance issues. I'm still in conversations - but plan on releasing the next version of dependency-check tomorrow morning. Hopefully, the release will resolve most issues although we may still have an issue with proxies that require authentication.

That is awesome news! Thank you so much!

hoeller commented 11 months ago

Has someone already tried running 9.0.1 with nvd.datafeedUrl set to a local cache? I guess this circumvents the ubiquitous problem because ODC does not have to connect to NVD and the cache can be obtained without virtualMatchString.

When I do this, I get two other errors.

Error downloading NVD CVE - https://redacted.com/nvd/cache/nvdcve-2012.json.gz Reason: READ_DATE_TIMESTAMPS_AS_NANOSECONDS
Error downloading NVD CVE - https://redacted.com/nvd/cache/nvdcve-2011.json.gz Reason: READ_DATE_TIMESTAMPS_AS_NANOSECONDS

> Task :dependencyCheckUpdate FAILED
10 actionable tasks: 1 executed, 9 up-to-date
The execution of the download was interrupted
org.owasp.dependencycheck.data.update.exception.UpdateException: The execution of the download was interrupted
    at org.owasp.dependencycheck.data.update.NvdApiDataSource.processDownload(NvdApiDataSource.java:261)
    at org.owasp.dependencycheck.data.update.NvdApiDataSource.processDatafeed(NvdApiDataSource.java:156)
    at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:108)
    at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:904)
    at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:876)
    at org.owasp.dependencycheck.gradle.tasks.Update.update(Update.groovy:56)
    at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
    at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:248)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:73)
    at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:233)
    at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:216)
    at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:199)
    at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:166)
    at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:105)
    at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:44)
    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:59)
    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:56)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:78)
    at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:56)
    at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:44)
    at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:67)
    at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:37)
    at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41)
    at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74)
    at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
    at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:50)
    at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:28)
    at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.executeDelegateBroadcastingChanges(CaptureStateAfterExecutionStep.java:100)
    at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:72)
    at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:50)
    at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:40)
    at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:29)
    at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:179)
    at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:70)
    at org.gradle.internal.Either$Right.fold(Either.java:175)
    at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59)
    at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:68)
    at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:46)
    at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36)
    at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25)
    at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36)
    at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22)
    at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:91)
    at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:55)
    at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364)
    at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55)
    at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:37)
    at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:65)
    at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:36)
    at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
    at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
    at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:77)
    at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:38)
    at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:108)
    at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:55)
    at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:71)
    at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:45)
    at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNonEmptySources(SkipEmptyWorkStep.java:177)
    at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:81)
    at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:53)
    at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32)
    at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21)
    at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
    at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:36)
    at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:23)
    at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:75)
    at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:41)
    at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.lambda$execute$2(ExecuteWorkBuildOperationFiringStep.java:66)
    at java.base@17.0.7/java.util.Optional.orElseGet(Optional.java:364)
    at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:66)
    at org.gradle.internal.execution.steps.ExecuteWorkBuildOperationFiringStep.execute(ExecuteWorkBuildOperationFiringStep.java:38)
    at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:32)
    at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:293)
    at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30)
    at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:21)
    at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37)
    at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27)
    at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:47)
    at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:34)
    at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:64)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:145)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:134)
    at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74)
    at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:78)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
    at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:42)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:331)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:318)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.lambda$execute$0(DefaultTaskExecutionGraph.java:314)
    at org.gradle.internal.operations.CurrentBuildOperationRef.with(CurrentBuildOperationRef.java:80)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:314)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:303)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:463)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:380)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
    at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:47)
    at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.concurrent.ExecutionException: java.lang.NoSuchFieldError: READ_DATE_TIMESTAMPS_AS_NANOSECONDS
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at org.owasp.dependencycheck.data.update.NvdApiDataSource.processDownload(NvdApiDataSource.java:251)
    ... 133 more
Caused by: java.lang.NoSuchFieldError: READ_DATE_TIMESTAMPS_AS_NANOSECONDS
    at com.fasterxml.jackson.datatype.jsr310.deser.InstantDeserializer._withFormatOverrides(InstantDeserializer.java:298)
    at com.fasterxml.jackson.datatype.jsr310.deser.JSR310DateTimeDeserializerBase.createContextual(JSR310DateTimeDeserializerBase.java:121)
    at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:830)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:550)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:294)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:638)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:539)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:294)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:615)
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:188)
    at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:28)
    at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:830)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:550)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:294)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:648)
    at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4861)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4731)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3677)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3645)
    at org.owasp.dependencycheck.data.update.nvd.api.DownloadTask.call(DownloadTask.java:95)
    at org.owasp.dependencycheck.data.update.nvd.api.DownloadTask.call(DownloadTask.java:42)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    ... 3 more

And also this.

Unable to find method ''boolean com.fasterxml.jackson.databind.util.NativeImageUtil.isInNativeImage()''
'boolean com.fasterxml.jackson.databind.util.NativeImageUtil.isInNativeImage()'

I tried this on several systems with an empty gradle cache. There seems to be some confusion about the Jackson version when parsing the JSON.

The first issue above regarding READ_DATE_TIMESTAMPS_AS_NANOSECONDS seems to be related to wrong configuration of the time module Jackson uses. I am only able to find very old discussions about this exact problem.

The second issue seems to be more recent, because the missing method NativeImageUtil.isInNativeImage() was introduced in databind v2.16.0. This is required by dependency-check-parent, but somehow gradle does not provide it? I'm running gradle 8.4 btw.

jay-bourland commented 11 months ago

FWIW - I was just able to run a full download of the database with v9.0.1. No errors were reported (other than the jcs stuff). Performance was comparable to downloading with v8.4.2. Twenty-four hours ago, I couldn't even get to the first progress report at 10,000 records.

The28AWG commented 11 months ago

How long does it take to load the database? Loading 1 hour. The size of the database is growing, which indicates that some work is being done. I can’t understand, is this a personal limitation due to the use of API keys, or is my karma so bad and I ended up on such a weak data transfer channel?

jeremylong commented 11 months ago

I wouldn't bother until the next release. I've been working with the engineers at the NVD. I should be releasing an updated version tomorrow morning.

The28AWG commented 11 months ago

Thanks!

marcelhdl commented 11 months ago

@jeremylong Works for me ! Thx for your great work!

OrangeDog commented 11 months ago

9.0.2 is working! No retries logged. Took ~10 minutes.

jeremylong commented 11 months ago

If you receive a 403 response - your API key is likely invalid/incorrect. See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

pnsag commented 11 months ago

9.0.2 worked even though updating took around 1,5 hours using apiKey

jeremylong commented 11 months ago

I just completed an update and the entire update took ~10 minutes.

gbrinkmann commented 11 months ago

I just completed an update and the entire update took ~10 minutes.

Initial downloads (first time user of 9.x) may take a while (currently about 60% in about 30min on dev notebook, full sonar3 build including your 9.0.2). No API key or other custom params. Okay for me since this is just a test. I don't mind durations in Bamboo sonar3 builds, initial build may be slow here too and updates may be slower than before 9.0.0 - would be okay for me.

Maybe we'll request an API Key - maybe not - we'll see :)

Thanks for your quick update!

jeremylong commented 11 months ago

All - if you are using the report format ALL or GitLab you may run into #6187. Sorry about that, expect a patch early next week (and please no need to "me too" on this issue).

jeremylong commented 11 months ago

Closing this issue as I believe the root of this problem is resolved now. Pending fixes in the 9.0.0 release are:

  1. Proxy authentication (#6127)
  2. report format ALL/GitLab (#6187)