aboutcode-org / scancode-toolkit

:mag: ScanCode detects licenses, copyrights, dependencies by "scanning code" ... to discover and inventory open source and third-party packages used in your code. Sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase, the Google Summer of Code, Azure credits, nexB and others generous sponsors!
https://github.com/aboutcode-org/scancode-toolkit/releases/
2.11k stars 545 forks source link

packagecode: gradle nebula dependency lock parser #2761

Open tardyp opened 2 years ago

tardyp commented 2 years ago

Short Description

gradle is a build tool for the ajva ecosystem especially popular in android apps.

nebula gradle dependency lock allows developer to lock their gradle dependencies and sub dependencies to specific versions packagecode should be able to parse this simple information

Possible Labels

Select Category

Describe the Update

lock format is documented here:

https://github.com/nebula-plugins/gradle-dependency-lock-plugin/wiki/Usage#lock-file-format

It is a simple json format that clearly shows the packages and versions. Should be quite easy to parse

How This Feature will help you/your organization

This will have using scancode to extract our gradle dependencies.

Possible Solution/Implementation Details

Example/Links if Any

Can you help with this Feature

Renault will probably contribute this feature in the next month

pombredanne commented 2 years ago

@tardyp note that I have done quite a bit of research on how to parse gradle builds at least the Groovy kind, and we could likely consider the Kotlin kind too

tardyp commented 2 years ago

@pombredanne That particular request is about nebula kind of package locks.

nebula looks like the poetry of gradle. Cool kids use it instead of the default gradle pinning method. I don't think it is really about kotlin, but just as Poetry.lock is another file format as requirement.txt or Pipfile.lock.

Big advantage I see is that it is a lockfile and there is no need to run the gradle toolchain to extract the info. It is just a json file.

pombredanne commented 2 years ago

@tardyp FYI @JonoYang is contributing some support for gradle in #2822

pombredanne commented 2 years ago

I think we should also support first the standard Gradle lockfile: https://docs.gradle.org/current/userguide/dependency_locking.html

Each line still represents a single dependency in the group:artifact:version notation It then lists all configurations that contain the given dependency Module and configurations are ordered alphabetically, to ease diffs The last line of the file lists all empty configurations, that is configurations known to have no dependencies

# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
org.springframework:spring-beans:5.0.5.RELEASE=compileClasspath, runtimeClasspath
org.springframework:spring-core:5.0.5.RELEASE=compileClasspath, runtimeClasspath
org.springframework:spring-jcl:5.0.5.RELEASE=compileClasspath, runtimeClasspath
empty=annotationProcessor
tardyp commented 2 years ago

Indeed... FWIW on my side we did not implement nebula after learning that Nebula is not anymore supported in more recent version of Android (as I was told).

We currently generate the dependency tree manually at project milestones :( Good news that gradle now has standard dependency locking, I will ping my devs if they can use that.

sschuberth commented 1 year ago

I think we should also support first the standard Gradle lockfile: https://docs.gradle.org/current/userguide/dependency_locking.html

Indeed. I believe this should be closed in favor of only supporting the Gradle built-in dependency locking mechanism.