prepDev Task does not add transitive dependencies to libraryDependencies.txt

[fmagin]

Disclaimer First off, this might also count as a feature request because I am straying far from the officially recommended development setup. This only seems to be an issue if developing an extension with a development setup that has further dependencies, and I haven't checked if this is even an issue with Eclipse, I am basically using raw java invocations to start Ghidra.

Describe the bug

prepDev does not add transitive dependencies to libraryDependencies.txt

To Reproduce

Setup a dev install per the README

Add a new Extension to Ghidra/Extensions that contains a build.gradle that specifies an external dependency

plugins {
    id "java"
}

repositories {
    mavenCentral()
    jcenter()
}

dependencies {
    implementation project(':Base')
    compile('org.jetbrains.kotlinx:kotlin-jupyter-kernel:0.8.3.283'){ exclude group: "org.slf4j", module: "slf4j-api" }
    compile group: 'org.json', name: 'json', version: '20210307'
}

run prepDev again.

The libraryDependencies.txt will contain the lines where GhidraJupyterKotlin :

Module: GhidraJupyterKotlin
    /home/fmagin/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlin-jupyter-kernel/0.8.3.283/70438b0e8955b3978cc0d2cf65573f2fb25bb3a5/kotlin-jupyter-kernel-0.8.3.283.jar
    /home/fmagin/.gradle/caches/modules-2/files-2.1/org.json/json/20210307/528c8df3757e8bcf151817b1d7b9b434f6aa08d5/json-20210307.jar

AllExternalLibs.txt will also not contain any transitive dependencies

Expected behavior

That the transitive dependencies are correctly recognized as part of prepDev and then added to libraryDependencies.txt so they will be available at runtime without having to specify all of them inside the build.gradle directly.

Additional context I recognize that this is somewhat of a niche problem and might not be an issue when using the official GhidraDev environment in Eclipse.

I think the problematic code is: https://github.com/NationalSecurityAgency/ghidra/blob/31b3db431e7dd8c4ecac54d95e1608b996a26989/build.gradle#L308-L345

all compile and runtime dependencies are checked for and added, but there is no check if they require additional dependencies again.

When building this plugin for a stable release I use a separate build.gradle that has a dedicated task that automagically copies all the relevant dependencies .jars into the lib folder, which is then simply included in the resulting .zip.

https://github.com/GhidraJupyter/ghidra-jupyter-kotlin/blob/e7f2ec2f8e66d6b1d72d29b81430f3e06d7173fd/GhidraJupyterKotlin/build.gradle#L53-L56

So the logic for this has to exist somewhere in Gradle and instead of copying them, the paths could just be added to the libraryDependencies.txt.

[fmagin]

The workaround I am using for now is changing getExternalRuntimeDependencies to this:

List<String> getExternalRuntimeDependencies(Project project) {
    List<String> list = new ArrayList<String>()

    list.addAll(getExternalDependencies(project.configurations.compile));
    list.addAll(getExternalDependencies(project.configurations.runtime));
    if (project.configurations.hasProperty('requiredLibs')){
        list.addAll(project.configurations.requiredLibs.collect {it.toString()});

    }
    return list
}

and declaring my dependencies like this:

dependencies {
    implementation project(':Base')
    compile group: 'org.jetbrains.kotlinx', name: 'kotlin-jupyter-kernel', version: '0.8.3.283',
            { exclude group: "org.slf4j", module: "slf4j-api" }
    compile group: 'org.json', name: 'json', version: '20210307'

    requiredLibs group: 'org.jetbrains.kotlinx', name: 'kotlin-jupyter-kernel', version: '0.8.3.283'
}

if I am changing list.addAll(getExternalDependencies(project.configurations.runtime)) to list.addAll(project.configurations.runtime.collect { it.toString()}) the libraryDependencies.txt gets way larger and will contain a ton of duplicates, so I am not sure if that has unintended effects.

I do not know what the requirements of this task and file are overall (I am familiar with the GhidraLauncher code that uses it though), so it is hard for me to judge how this should be changed, but if you can tell me what is needed, I can create a PR for this.

[ghidra1]

We require explicit compile or runtime dependency declarations of all dependencies within the build.gradle file. For modules which introduce such a jar dependency an appropriate IP entry is required within the corresponding Module.manifest file. This is done to strictly manage the introduction of jars into the application as well as ensuring that appropriate licensing (i.e. IP) has been properly reviewed and tagged.

We use compile or runtime dependency declarations for all required jars.