Task get stuck because of transitive dependencies

gregoryboue commented 10 months ago

Describe the bug

In an android project the generateDepTrees task get stuck or lead to java heap space error.

The behaviour seems happens only when there is conflict between dependency and transitivity dependencies.

Current behavior

When i run /gradlew clean generateDepTrees the task can be very long (more than 10 minutes) and end with this error

failed while building 'gradle' dependency tree:
error running gradle-dep-tree: exit status 1
Exception in thread "Daemon periodic checks" java.lang.OutOfMemoryError: Java heap space

The debug log shows :

[DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
[DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
[DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
[DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.

Reproduction steps

Have a very simple gradle android project with gradle 8+ and the following dependencies in build.gradle :

implementation 'com.google.android.material:material:1.10.0'
implementation 'androidx.navigation:navigation-fragment:2.5.3'

Launch ./gradlew clean generateDepTrees.... command and then the problem will happen.

Notes :

If androidx.navigation:navigation-fragment:2.5.3 dependency is removed, then then it's ok.
If com.google.android.material:material dependency is downgrade to 1.9.0, then it's ok (because in 1.10.0 material add a dependency toward androidx.activity:1.8.0.

if i exclude androidx.activity from material, it's ok :

implementation('com.google.android.material:material:1.10.0') {
 exclude group: 'androidx.activity', module: 'activity'
}
implementation 'androidx.navigation:navigation-fragment:2.5.3'

Expected behavior

Functionnal generateDepTrees output

Gradle Dependency Tree version

3.0.1

Operating system type and version

Ubuntu-jammy, windows 11

asafgabai commented 10 months ago

Hi @gregoryboue, thanks for reporting this! I couldn't replicate the issue, but in past versions of gradle-dep-tree, we encountered similar problems. If you're using our provided init.gradle, I suggest updating it with the following line to make sure you use the latest version:

        classpath 'com.jfrog:gradle-dep-tree:3.0.1'

Feel free to let me know if this resolves the issue or if you need further assistance.

gregoryboue commented 10 months ago

Hi @asafgabai ,

Thanks for the answer, i can confirm that it's ok with 3.0.1 and KO with 3.0.0, i will wait that 3.0.1 version to be embed in jfrog-cli-core because currently jf audit doesn't work for my project

asafgabai commented 10 months ago

@gregoryboue Great! it was upgraded to 3.0.1 in this pull request and will be released soon.

gregoryboue commented 10 months ago

Hi,

The JFrog CLI 2.52.0 with gradle-dep-tree 3.0.1 has been released today, with my sample project, i still have problem with jf audit --gradle command :

jf audit --gradle
[Debug] JFrog CLI version: 2.52.0
[Debug] OS/Arch: linux/amd64
[Debug] Sending HTTP GET request to: https://myartifactory/xray/api/v1/system/version
[Debug] Usage Report: Sending info...
[Debug] Sending HTTP GET request to: https://myartifactory/artifactory/api/system/version
[Debug] Artifactory response: 200 OK
[Debug] JFrog Artifactory version is: 7.55.10
[Debug] Sending HTTP POST request to: https://myartifactory/artifactory/api/system/usage
[Debug] Sending HTTP GET request to: https://myartifactory/xray/api/v1/entitlements/feature/contextual_analysis
[Debug] The path '/home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/.gitignore' is excluded
[Debug] mapped 1 working directories with indicators/descriptors:
 {
   "/home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE": [
     "/home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/build.gradle"
   ]
 }
[Debug] Detected 1 technologies at /home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE: [gradle].
[Info] Preforming 1 SCA scans:
[
   {
     "Technology": "gradle",
     "WorkingDirectory": "/home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE",
     "Descriptors": [
       "/home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/build.gradle"
     ]
   }
]
[Info] Running SCA scan for gradle vulnerable dependencies in /home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE directory...
[Info] Calculating Gradle dependencies...
[Debug] Using resolver config from /home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/.jfrog/projects/gradle.yaml
[Debug] Preparing to read the config file /home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/.jfrog/projects/gradle.yaml
[Debug] Found resolver in the config file /home/jenkins/agent/workspace/groupeeul-dosn/02608/tnr/TEMPLATE_ANDROID_SNAPSHOT_TU_GBE/.jfrog/projects/gradle.yaml
[Debug] JFROG_CLI_RELEASES_REPO is not set
[Debug] The project dependencies will be resolved from https://myartifactory/artifactory/ from the maven_02608 repository
[Info] Running gradle deps tree command: ./gradlew clean generateDepTrees -I /tmp/jfrog.cli.temp.-1701724567-540383840/gradledeptree.init -q -Dcom.jfrog.depsTreeOutputFile=/tmp/jfrog.cli.temp.-1701724567-540383840/gradledeptree.out -Dcom.jfrog.includeAllBuildFiles=true
Killed

The exit code is 137. When i just run generateDepTrees command with custom init.gradle to use gradle-dep-tree 3.0.1 it works.

gregoryboue commented 9 months ago

@asafgabai Did you reproduce the behaviour with jfrog CLI 2.52.0 ?

asafgabai commented 9 months ago

@gregoryboue The issue was reproduced. The team is working on fixing it.

gregoryboue commented 9 months ago

thanks for the answer

eranturgeman commented 9 months ago

Hello @gregoryboue,

I wanted to give you an update on our progress. The concerns surrounding gradle-dep-tree have already been resolved. Regarding 'jf audit' with Gradle, I've recently opened a PR to begin addressing this specific issue. I'll ensure to keep you updated on any developments or future updates.

gregoryboue commented 9 months ago

Thanks you a lot @eranturgeman

eranturgeman commented 9 months ago

Hello @gregoryboue! I'm pleased to share that the bug fix has been released! I'll keep this issue page open for a while longer, so please don't hesitate to respond here if you encounter any additional issues. If the fix resolves your problem, your feedback confirming its success would be greatly appreciated.

gregoryboue commented 9 months ago

I can confirm this issue was fixed

jfrog / gradle-dep-tree