Dependency graph support for Gradle

[zzzowoey]

Dependency graphs only support Maven right now. If a Java project doesn't contain a pom.xml, it won't get a dependency graph, related insights, or security alerts at all. Gradle is perfectly compatible with Maven dependencies, so I don't see why Gradle support isn't available (at least for Maven artifacts).

This is a feature that has existed for 2 years and it's limited to 5 languages and a tiny handful of build systems for each. It goes without arguing that dependency graphing needs expansion.

[mcabezas]

Hi everyone, is it possible to work on it as a contributor?

[jhutchings1]

👋 Hi folks, we don't have a contribution model for new dependency graph ecosystems, but we are working to expand the set of ecosystems right now. Gradle is on our list, so stay tuned!

[zzzowoey]

Thanks for the update. Glad to hear it.

[plokhotnyuk]

sorry for the off topic... @jhutchings1 is sbt (Scala Build Tool) in your list?

[jhutchings1]

@plokhotnyuk Let me answer your question more generally because I'm sure everyone has an ecosystem they'd like to see 😄

We're prioritizing dependency graph ecosystems by tracking the commit volume for particular programming languages, as well as looking at the maturity of the package management ecosystem. The Octoverse reports should give you a good idea of how different languages stack up. From there, we've got a deep backlog and are working through the ecosystems one by one.

[dagguh]

Gradle build files are not declarative, they're scripts. In general, you need to run gradle dependencies to get the actual dependency trees. However, if a repo uses dependency locks, then it's fully declarative and static, which should make it easier and safer to ingest.

[firmsoil]

Gradle is on our list, so stay tuned!

Any update on this please?

[jhutchings1]

Tagging my colleague @iamwillbar who is the best person to speak to this. We reprioritized or dependency graph investments this fall.

[ghost]

I am so glad this is getting attention! We continue to have a need for this functionality.

[bsautner]

would it work if we wrote a gradle script that outputed a fake pom.xml in maven format - if the file was just sitting there getting updated from our build but not actually part of our build process, github could ingest it for vulnerability scans - we really want this for android projects

[jhansche]

would it work if we wrote a gradle script that outputed a fake pom.xml in maven format - if the file was just sitting there getting updated from our build

I can confirm this does work. We have a CI job that runs with a simple Gradle init script against all of our Android projects (libraries and apps, including subproject modules) that enables the maven publish plugin and generates a POM for each project and module, then collects all those into a temporary project that consists only of POM files. GitHub is correctly detecting all the POM files in that temporary repo.

[rocketraman]

According to Configuring GitHub Dependabot security updates this should already work for Gradle, with the appropriate dependabot configuration.

It should look something like this (disclaimer, I haven't tried this myself yet):

$ cd my-project
$ mkdir .github
$ cat > .github/dependabot.yaml
version: 2
updates:
  - package-ecosystem: "gradle"
    # path to build.gradle
    directory: "/whatever"
    schedule:
      interval: "daily"

Note that Gradle is not explicitly listed as supported but it is listed in the reference docs for package-ecosystem.

UPDATE: I tried it and it does work, sort of -- dependabot reads the Gradle configuration and submits pull requests. However, the dependency graph "Dependencies" and "Dependents" tabs are not, at least yet, updated with Gradle information. Also, security alerts are generated from the Dependencies/Dependents information, and this approach therefore does not generate security alerts.

[jhansche]

would it work if we wrote a gradle script that outputed a fake pom.xml in maven format - if the file was just sitting there getting updated from our build

I can confirm this does work. We have a CI job that runs with a simple Gradle init script...

For reference, the init-script can be as simple as:

allprojects { p ->
    afterEvaluate {
        if (p.plugins.hasPlugin("maven-publish")) return
        p.apply plugin: "maven-publish"
    }
}

Some additional changes might be needed, such as adding the expected publications - for example:

publishing.publications.create("release", MavenPublication) {
    from component // e.g., components.java; or an Android project's component if you're using a modern AGP
}

And executed with:

./gradlew -I maven-publish.init.gradle generatePomFile

It will output all POM files to $module/build/publications/*/pom-default.xml for each module.

[DevCharly]

I'm also interested in this feature, especially in the "Dependents" tab ("Insight > Dependency graph > Dependents").

Surprisingly there are already some Gradle projects where the "Dependents" tab is filled:

• https://github.com/spring-projects/spring-framework/network/dependents
• https://github.com/mockito/mockito/network/dependents
• https://github.com/cbeust/testng/network/dependents

Anybody know how they did this?

Interestingly the "Dependencies" tab of these project says "No manifest files found".

The only information that GitHub actually needs to fill the "Dependents" tab is the Maven groupID and artifactID(s).

[slawekjaranowski]

There is my post https://github.community/t/insights-dependency-graph-dependents-for-gradle-projects/148578 Nobody answers

[aviadsTraiana]

would it work if we wrote a gradle script that outputed a fake pom.xml in maven format - if the file was just sitting there getting updated from our build

I can confirm this does work. We have a CI job that runs with a simple Gradle init script...

For reference, the init-script can be as simple as:

allprojects { p ->
    afterEvaluate {
        if (p.plugins.hasPlugin("maven-publish")) return
        p.apply plugin: "maven-publish"
    }
}

Some additional changes might be needed, such as adding the expected publications - for example:

publishing.publications.create("release", MavenPublication) {
    from component // e.g., components.java; or an Android project's component if you're using a modern AGP
}

And executed with:

./gradlew -I maven-publish.init.gradle generatePomFile

It will output all POM files to $module/build/publications/*/pom-default.xml for each module.

Hi, not sure I understand how that works.. that means that those build files need to be available in my repo?

[rkrisztian]

No need for the if statement though. You can apply a plugin twice, but will be loaded only once, see https://discuss.gradle.org/t/issue-a-warning-if-the-same-plugin-is-double-applied-to-a-project/404

[jhansche]

Hi, not sure I understand how that works.. that means that those build files need to be available in my repo?

Generally yes. In my case, I have a CI (Jenkins) job that just writes that init-script out to a file before calling the gradlew command-line with -I. The init script gets applied to my project, which applies maven-publish and creates the publication(s), and the task generates the POM files. After that I copy the pom-default.xml outputs to a new directory structure, renamed to pom.xml for each module. That new directory structure is a separate git project that contains only pom files, but you could also just commit them back to the same project if you don't want to keep them separated (just means that there will be automated commits in the repo). That new repo of pom.xml files is picked up correctly by github dependencies.

[duckAsteroid]

Committing generated pom.xml files still seems like a workaround to a basic lack of function in GitHub. And it's getting old now; this issue is nearly two years old, and the OP said the lack of support was two years old when created. 4 years on and no action on what is (apparently) now the most popular Java build tool (though its close) : https://www.jrebel.com/blog/2020-java-technology-report [see section "Most Popular Java Build Tool"]

[duckAsteroid]

NB: In the 2021 version of that report (I can only find a private PDF download) it's dropped to 20% vs. Maven 67%.

[rocketraman]

Querying the free github_repos dataset in BigQuery:

SELECT COUNT(DISTINCT(project_name))
  FROM (
  SELECT
    ARRAY_REVERSE(SPLIT(repo_name, '/'))[SAFE_OFFSET(0)] AS project_name  -- avoid forks
  FROM
    `bigquery-public-data.github_repos.files`
  WHERE
    --path = 'build.gradle' OR path = 'build.gradle.kts'
    path = 'pom.xml'
  GROUP BY
    project_name
)

shows:

64,864 repos containing build.gradle or build.gradle.kts 83,643 repos containing pom.xml.

Obviously, some will contain both, but in general, it looks like Gradle is not quite as popular as Maven, but its close. Whether Gradle is the first or second most used build tool for the second or third most used language in the world (not even counting Kotlin, Scala, and other JVM projects that use it) seems somewhat academic though... its clearly popular enough for first-party support.

[Hakky54]

@jhutchings1 and @iamwillbar any updated from your side? We are really looking forward on this feature 😊

[jhutchings1]

@Hakky54 Thanks for your continued patience on this. We're aware this is still a painpoint for the Gradle community. I don't have a shareable timeline just yet, but please keep an eye on https://github.com/github/roadmap. We'll open a roadmap item as support for it comes nearer.