Fully Featured Gradle Support with the Gradle Tooling API

[JLLeitschuh]

Foreward I'm not intimately familiar with how Dependabot works so please excuse me if I wildly misrepresent something about Dependabot.

Problem

Currently, Dependabot is attempting to parse build.gradle files as text files instead of fully understanding the complexities of Gradle Builds.

This is because the dependencies declared in a build.gradle file doesn't tell the full story.

Additionally, since Gradle now supports the Gradle Kotlin DSL, there are now build.gradle.kts files. On top of that, users can also change the name of their build files so that they call the files my-project.gradle.kts and Gradle will handle that fine.

This current methodology completely misses any dependencies added by plugins, transitive dependencies, and dependencies added using Gradle in abnormal but valid ways.

Fundamentally this issue arises because unlike NPM and Yarn, where dependency declaration files are static, Gradle & Maven both require code execution to fully resolve the dependency graph.

Solution

Gradle offers something called the Gradle Tooling API. This tooling API is already in use in IntelliJ, Eclipse, and the Gradle Test Kit.

The Tooling API exposes the entire dependency graph for the repositories build in a way that can be transformed into a format Dependabot can understand.

The Tooling API executes the Gradle Build and can be used to extract metadata about the build for consumption by external tools.

Risks

There are certain risks that are taken on by moving forward with this plan.

Security

In order to use the Tooling API, Dependabot will be required to execute the user's code in their repository. Fundamentally, this introduces untrusted code execution into the same environment where Dependabot is being executed. I don't know what the security model of Dependabot's execution is, but if executing user code is a non-starter, using the Tooling API won't work.

Performance

Sometimes developers have really, really weird builds. Sometimes they also have really slow builds. Calling the ToolingAPI will bring the Dependabot execution time from being predictable with respect to the number of files being parsed, to a non-deterministic amount of execution time.

Enviromental

Certain versions of Gradle only work with certain versions of the JVM. Certain builds only work with certain versions of the JVM. Implementing this feature will require maintainers of repositories to explicitly declare what version of the JVM their build depends upon.

Rewards

Keeping the Java Ecosystem Safe.

Using the GitHub search functionality for filename:gradle-wrapper.jar returns 2.55 million results. Additionally, Gradle is the official build tool for the Android Ecosystem.

Having good tooling support around Gradle from GitHub and Dependabot would protect developers, corperations, and Android users around the world.

[3flex]

The plugin mentioned above is now integrated with the Gradle build action and the feature is considered stable since v2.7.0: https://github.com/gradle/gradle-build-action/releases/tag/v2.7.0

[jonjanego]

Hello to the many watchers, just amplifying this a bit, the Gradle team has merged this (and other) functionality into a single setup-gradle action

[huehnerlady]

@jonjanego where does this github action check for new dependencies? I couldn't see anything in the readme

[gredler]

Note that you may need to disable the Gradle caching (cache-disabled: true) if you're also using codeql-action, otherwise the CodeQL step will fail when there is nothing to compile (because the compile results were cached in the last run).

[jonjanego]

@huehnerlady - this section of the docs talks about what i was referring to. the short version is, after running a gradle build, if you enable the option to do dependency submission, that will submit the dependencies to GitHub for us to analyze. This will build a far more complete version of the dependency graph than we'd be able to do without submitting them. Its impact onto dependabot usage is that you'll be getting a more complete list of packages with security updates available.

I'm sure that the gradle team would be happy to discuss more! I'll point them to this thread to see if they'd like to chime in.