Gradle build not working with any dependency

[blvp]

build.gradle

buildscript {
    repositories {
        mavenCentral()
    jcenter()
    }
}
apply plugin: 'application'
apply plugin: 'java'

repositories {
    mavenCentral()
    jcenter()
}
mainClassName = 'JCublasSample'
dependencies {
    compile group: 'org.jcuda', name: 'jcuda', version: '0.8.0'
    compile group: 'org.jcuda', name: 'jcublas', version: '0.8.0'
}

Error: Could not find jcuda-natives-${jcuda.os}-${jcuda.arch}.jar (org.jcuda:jcuda-natives:0.8.0). Seems like gradle doesn't support properties in the dependencies pom files. I tried:

1. add gradle.properties with jcuda.os and jcuda.arch properties
2. add to build.gradle System.setProperty(...) in buildscript

But none of them works. I know that it is not jcuda issue that gradle does not support properties in dependent pom files, but it is not usable with this build system.

I really don't want to use maven.

[jcuda]

Gradle is so commonly used that it's good to know when there's an issue with that. Although I have to admit that I never used gradle (beyond using it, as in executing gradlew ...). This is also why I'm not entirely sure what the key point is here.

How I understood it:

The properties that are set in the jcuda-parent POM file are only applied "locally", but are not "visible" in the POM files of the <dependencies> - is that right?

Then I wonder whether there is a similar concept (or rather: any way at all) to pass such information from one parent POM to a child POM. Are the approaches (1. and 2.) that you mentioned common to achieve something like that? Where exactly did you add this information, and how is gradle supposed to read and use this information? (Sorry, these may be dumb questions, and likely, I could find the answer with diligent googling, but in order to quickly get a grip on this issue, having a short summary may be helpful)

[blvp]

I found one discussion about this problem.

Where exactly did you add this information, and how is gradle supposed to read and use this information?

1. create file gradle.properties in project root with content:

   jcuda.os=apple
   jcuda.arch=x86_64

2. modify build.gradle add this lines:

   System.setProperty('jcuda.os', 'apple')
   System.setProperty('jcuda.arch', 'x86_64')

But this approaches are not working since gradle won't support this feature.

I think there might be solution for this kind of multiplatform distribution. One way:

1. generate jars for every architecture and os that is supported
2. add qualifying name for it (eg. org.jcuda:jcuda-win-x86_64:0.8.0)
3. add platfrom specific dependencies at users builds in manner of this article

Or other:

1. add classifier for builds (eg. <classifier>win-x86_64</classifier>
2. gradle supports classifiers, but still users should define proper classifier of architecture and os family (looking for gradle plugin)

I'm not in common with multi platform software distribution, so my solutions should be checked.

[blvp]

I read thread in #14 and all my proposed solutions seems dumb now. I'll try to dig into this problem further.

[jcuda]

Yes, the Maven depolyment and the native libraries always are a source of headaches. But I hope that with the current solution, they are no longer a source of headaches for the user of JCuda, but only for me ;-)

Any insights or hints about how the gradle build files could look like are welcome.

However, since gradle is supposed to be (far!) more powerful than Maven, I think that there should be a solution, and it should not be sooo complicated. I'd really like to add some gradle build files to JCuda - again, to reduce the hassle for others - but first have to become more familiar with gradle, and actually try it out.

From some of your links and related websearch results, I think that it should be possible to do something that looks roughly (VERY roughly!) like this:

def osString = osString(); // See note below
def archString = archString(); // See note below

dependencies {
    compile group: 'org.jcuda', name: 'jcuda', version: '0.8.0'
    compile group: 'org.jcuda', name: 'jcuda', classifier: 'natives-'+osString+'-'+archString, version: '0.8.0'
    compile group: 'org.jcuda', name: 'jcublas', version: '0.8.0'
    compile group: 'org.jcuda', name: 'jcublas', classifier: 'natives-'+osString+'-'+archString, version: '0.8.0'
}

The osString() and archString() functions are taken from the LibUtils class, but it should be possible to implement them similarly in the gradle build file.

But in fact, this nearly looks too easy to be true...

Do you think that this might work? (I'll have to allocate some time to actually try it out...)

---

EDIT: A side note: What usually is accomplished with these profiles in the Maven POM is basically: Establishing the dependency from the actual JAR to the natives JAR. So when you add jcuda.jar as a dependency, then it will (depending on your OS/architecture) add the respective jcuda-natives-os-arch.jar as a dependency. This automatation would not apply here, due to the lack of profiles in gradle. But I think that with proper gradle build files and these osString/archString utility methods, the process of adding a dependency (and the corresponding natives dependency) could still be comparatively easy.

[AlexandrTerentyev]

It works:

def classifier = osString()+'-'+archString()

compile (group: 'org.jcuda', name: 'jcuda', version: '0.8.0',){
    transitive=false
}
compile (group: 'org.jcuda', name: 'jcublas', version: '0.8.0'){
    transitive=false
}

compile group: 'org.jcuda', name: 'jcuda-natives', classifier: classifier, version: '0.8.0'
compile group: 'org.jcuda', name: 'jcublas-natives', classifier: classifier, version: '0.8.0'

[jcuda]

Great to hear that!

Admittedly, I didn't yet have a closer look at this (it got buried under more pressing tasks).

So you took the osString and archString as they are implemented in the LibUtils? I wonder whether (working) gradle files could be added. If you could provide these (via pull request, or in doubt, simply via mail) that would be great!

(I'll leave the issue open as a reminder (for me) that having gradle build files would be nice)

[evbarnett]

I know this is an old thread- but did you ever get the gradle files? Because I'd be glad to provide them if not. I'd like for this solution to be documented for clearly, because it was perplexing me. Thanks for the fix!

I had to make some slight modifications, because I could not access the osString() & archString() methods since the jar files for the non-natives were not downloaded yet.

I'm guessing it worked for him because he already had the partial download, but this solution would not work if the project was shared over version control- since they wouldn't have had the methods to call yet. The solution for me was to pull out the useful LibUtils file and add it to the classpath (this file being included in version control) before gradle loads external dependencies. (Hopefully this explanation made sense)

[jcuda]

Thanks, that would be great (I still haven't tried it out - I should probably increase the priority here, since it should not be toooo hard to get it running, given the information that is summarized here).

Regarding the LibUtils: The methods that are required are actually not very complicated - they are mainly convenience wrappers around some JVM-internal calls. Maybe the relevant methods (or particularly, the osString and archString) can be obtained where they are needed, without adding a dependency to the LibUtils class...?

[evbarnett]

Not sure why I didn't think of that, it makes it a lot cleaner!

Here is a gradle file that correctly pulls the dependencies. I tested it on Windows 10 64-bit with gradle v2.3.3:

apply plugin: 'java'

def static getOsString() {
    String vendor = System.getProperty("java.vendor");
    if ("The Android Project" == vendor) {
        return "android";
    } else {
        String osName = System.getProperty("os.name");
        osName = osName.toLowerCase(Locale.ENGLISH);
        if (osName.startsWith("windows")) {
            return "windows";
        } else if (osName.startsWith("mac os")) {
            return "apple";
        } else if (osName.startsWith("linux")) {
            return "linux";
        } else if (osName.startsWith("sun")) {
            return "sun"
        }
        return "unknown"
    }
}

def static getArchString() {
    String osArch = System.getProperty("os.arch");
    osArch = osArch.toLowerCase(Locale.ENGLISH);
    if ("i386" == osArch || "x86" == osArch || "i686" == osArch) {
        return "x86";
    } else if (osArch.startsWith("amd64") || osArch.startsWith("x86_64")) {
        return "x86_64";
    } else if (osArch.startsWith("arm64")) {
        return "arm64";
    } else if (osArch.startsWith("arm")) {
        return "arm";
    } else if ("ppc" == osArch || "powerpc" == osArch) {
        return "ppc";
    } else if (osArch.startsWith("ppc")) {
        return "ppc_64";
    } else if (osArch.startsWith("sparc")) {
        return "sparc";
    } else if (osArch.startsWith("mips64")) {
        return "mips64";
    } else if (osArch.startsWith("mips")) {
        return "mips";
    } else if (osArch.contains("risc")) {
        return "risc";
    }
    return "unknown";
}

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')

    // Add your other dependencies here:

    // JCuda dependencies are below

    def classifier = getOsString() + "-" + getArchString()

    // Set JCuda version here, or if multiple modules use JCuda, 
    // you should set a global variable like so:
    //
    // ext {
    //  jCudaVersion = "0.8.0"
    // }
    //
    // In your *top level* build gradle, and use
    // rootProject.ext.jCudaVersion instead of jCudaVersion when you need to access it

    def jCudaVersion = "0.8.0"

    // JCuda Java libraries

    compile(group: 'org.jcuda', name: 'jcuda', version: jCudaVersion) {
        transitive = false
    }
    compile(group: 'org.jcuda', name: 'jcublas', version: jCudaVersion) {
        transitive = false
    }
    compile(group: 'org.jcuda', name: 'jcufft', version: jCudaVersion) {
        transitive = false
    }
    compile(group: 'org.jcuda', name: 'jcusparse', version: jCudaVersion) {
        transitive = false
    }
    compile(group: 'org.jcuda', name: 'jcurand', version: jCudaVersion) {
        transitive = false
    }

    // JCuda native libraries

    compile group: 'org.jcuda', name: 'jcuda-natives',
            classifier: classifier, version: jCudaVersion
    compile group: 'org.jcuda', name: 'jcublas-natives',
            classifier: classifier, version: jCudaVersion
    compile group: 'org.jcuda', name: 'jcufft-natives',
            classifier: classifier, version: jCudaVersion
    compile group: 'org.jcuda', name: 'jcusparse-natives',
            classifier: classifier, version: jCudaVersion
    compile group: 'org.jcuda', name: 'jcurand-natives',
            classifier: classifier, version: jCudaVersion
}

[jcuda]

Thanks, I'll try it out ASAP. There are currently some higher-priority tasks in the queue, but just running this script should not be too much effort (if it works ;-)).

---

This should not yet be directly relevant for JCuda, but to give a slightly bigger picture:

One task in the queue is related to that: JOCL (one of my other projects) is supposed to be updated to work on Android (as of https://github.com/gpu/JOCL/pull/16 ). This implies some changes to the LibUtils architecture detection. The LibUtils class exists in both projects. And I know that it has been copied-and-pasted into other projects. I already considered to extract it into a own library, to avoid further duplications and diverging developments. A similar library already exists, namely https://github.com/scijava/native-lib-loader , but of course, this does not cover 100% but only 95% of the use-cases that I need for JOCL and JCuda.

---

In any case, if the script works as expected, I'd add it alongside the current build files, with a link to this issue and credits to you and @blvp

[jcuda]

Again, I'm not familiar with Gradle, and at the risk of sounding dumb:

So what is this now?

This build file does not do anything. What tasks should be added there? Should one such file be in each project? Or is this supposed to be used as a "template" for someone who wants to set up the dependency to JCuda in his gradle file?

[evbarnett]

This is a bare minimum build file - if someone has a new project and they would want to use Jcuda, they could use this as the basis and add additional dependencies where I marked it.

If someone wants to use Jcuda with an existing project, they need to copy and paste the two functions then add all the lines below "JCuda dependencies are below" into their dependencies block.

This gradle script is enough to compile a project- as you can see I don't need to list my java files or provide compilation instructions- gradle does that for me.

Gradle is modular, so there is a build file for each module. If you have multiple modules, and only one uses jcuda, you only need to include jcuda as a dependency for that module.

[jcuda]

I've heard many good things about gradle, and it indeed seems far more flexible than Maven. But I assumed that it would, for example, be possible to execute the unit tests or so. (There seems to be a "test" task, and also things like a deployment to the local maven repo, but ... the documentation is rather extensive, I'll probably have to play around with this "from scratch" in a small "Hello World" project).

So you think that if I add this to jcuda-main, people will know what to do with it? (I thought that there would then eventually be this magic gradlew thingy, that offers all the tasks that are mentioned in the documentation)

[jcuda]

So assuming that this build file will serve as a template, and people who are more familiar with Gradle will know how to use it, I added it at https://github.com/jcuda/jcuda-main/blob/master/USAGE.md#using-jcuda-with-gradle (and took the chance to refactor the README a bit).

I'm closing this for now, although the point to familiarize myself with Gradle is still on my TODO list, and suggestions on how to improve the template or its description are of course welcome.

Thanks again to @evbarnett and @blvp for setting this up!