Build failures with Jack compiler

[Aranda]

Has anybody got this working yet? It seems to build until I start to use a generated Model class in my code.

DeliverySpec.java

package com.me.testapp.squidb;

import com.yahoo.squidb.annotations.PrimaryKey;
import com.yahoo.squidb.annotations.TableModelSpec;

@TableModelSpec(className="Delivery", tableName="delivery")
public class DeliverySpec {
    @PrimaryKey
    public long delivery_id;
}

Usage of the generated model:

package com.me.testapp.wire;

import com.me.testapp.squidb.Delivery;

public class DeliveryWire {
    public long delivery_id;

    public static Delivery toModel(DeliveryWire dw)  {
        Delivery d = new Delivery();
        d.setDeliveryId(dw.delivery_id);
        return d;
    }

    public static DeliveryWire fromModel(Delivery d) {
        DeliveryWire dw = new DeliveryWire();
        dw.delivery_id = d.getDeliveryId();
        return dw;
    }
}

Project level build.gradle

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:2.2.0-beta1'
    }
}

allprojects {
    repositories {
        jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

Module level build.gradle

apply plugin: 'com.android.application'

android {
    compileSdkVersion 24
    buildToolsVersion "24.0.1"

    defaultConfig {
        applicationId "com.me.testapp"
        minSdkVersion 21
        targetSdkVersion 24
        versionCode 1
        versionName "1.0"
        jackOptions {
            enabled true
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:24.2.0'
    compile 'com.android.support:support-v4:24.2.0'
    compile 'com.google.android.gms:play-services-appindexing:9.4.0'

    compile 'com.yahoo.squidb:squidb:3.1.2'
    compile 'com.yahoo.squidb:squidb-annotations:3.1.2'
    compile 'com.yahoo.squidb:squidb-android:3.1.2'
    annotationProcessor 'com.yahoo.squidb:squidb-processor:3.1.2'
}

Initial build error:

* What went wrong:
Execution failed for task ':app:transformJackWithJackForDebug'.
> com.android.jack.ir.JNodeInternalError: Error building Jack IR: com.android.jack.eclipse.jdt.internal.compiler.ast.TypeDeclaration at "/Users/aranda/code/SignatureCapture/app/src/main/java/com/adapptor/signaturecapture/wire/DeliveryWire.java:9.14-9.25"

Stack trace that appears if you try to build again:

:app:transformJackWithJackForDebug
Switch enum optimization is disabled due to incremental compilation

ERROR: Library reading phase: file '/Users/aranda/code/SignatureCapture/app/build/intermediates/packaged/debug/classes.zip' is an invalid library

com.android.jack.api.v01.CompilationException: Library reading phase: file '/Users/aranda/code/SignatureCapture/app/build/intermediates/packaged/debug/classes.zip' is an invalid library
    at com.android.jack.api.v01.impl.Api01ConfigImpl$Api01CompilationTaskImpl.run(Api01ConfigImpl.java:113)
    at com.android.builder.core.AndroidBuilder.convertByteCodeUsingJackApis(AndroidBuilder.java:1821)
    at com.android.builder.core.AndroidBuilder.convertByteCodeUsingJack(AndroidBuilder.java:1648)
    at com.android.build.gradle.internal.transforms.JackTransform.runJack(JackTransform.java:220)
    at com.android.build.gradle.internal.transforms.JackTransform.transform(JackTransform.java:194)
    at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:177)
    at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:173)
    at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:156)
    at com.android.build.gradle.internal.pipeline.TransformTask.transform(TransformTask.java:172)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$IncrementalTaskAction.doExecute(AnnotationProcessingTaskFactory.java:245)

[Aranda]

In case it's useful, the SquidDatabase class:

package com.me.testapp.squidb;

import android.content.Context;
import android.util.Log;

import com.yahoo.squidb.android.AndroidOpenHelper;
import com.yahoo.squidb.data.ISQLiteDatabase;
import com.yahoo.squidb.data.ISQLiteOpenHelper;
import com.yahoo.squidb.data.SquidDatabase;
import com.yahoo.squidb.sql.Table;

public class    ModelDatabase extends SquidDatabase {
    private static final int VERSION = 1;

    private static ModelDatabase sInstance = null;

    private Context mContext;

    public static ModelDatabase getInstance() {
        if (sInstance == null){
            Log.e("ModelDatabase", "Call createInstance() before first use of getInstance()");
        }
        return sInstance;
    }

    public static void createInstance(Context context)
    {
        sInstance = new ModelDatabase(context);
    }

    private ModelDatabase(Context context) {
        super();
        mContext = context;
    }

    @Override
    public String getName() {
        return "test.db";
    }

    @Override
    protected int getVersion() {
        return VERSION;
    }

    @Override
    protected Table[] getTables() {
        return new Table[] {
                Delivery.TABLE
        };
    }

    @Override
    protected boolean onUpgrade(ISQLiteDatabase db, int oldVersion, int newVersion) {
        return true;
    }

    @Override
    protected ISQLiteOpenHelper createOpenHelper(String databaseName, OpenHelperDelegate delegate,
                                                 int version) {
        return new AndroidOpenHelper(mContext, databaseName, delegate, version);
    }
}

[sbosley]

We're not using Jack ourselves, but I believe the android-apt plugin that runs code generation is not compatible with Jack. You can see more details here: https://bitbucket.org/hvisser/android-apt/issues/33/support-for-jack-and-jill. That issue makes it sound like Jack includes its own support for annotation processors, but having never used it I don't know how to configure it. If you get it working let us know so we can test it out on our end and document what the correct configuration should be!

[sbosley]

Oh hmm never mind, I see you've already done that. Well, short answer then is I don't know and we'll have to look into it, but we probably won't be officially supporting Jack at least until its a stable release.

[Aranda]

Yeah, unfortunately reverting to android-apt with Jack results in this (by default - perhaps there are other apt options to get around it):

Error:Could not get unknown property 'classpath' for task ':app:transformJackWithJackForDebug' of type com.android.build.gradle.internal.pipeline.TransformTask.

The issue you linked seems to be a won't fix so we'll wait until Jack goes stable and SquiDB supports it before using them together.

[sbosley]

Yeah I think that will be best. Sorry I can't offer a better solution right now, but it's definitely on our radar now! Here's the workaround I'd suggest for you in the meantime: create a small, separate module just for your model specs and nothing else. You can compile that module with the non-Jack toolchain using android-apt and then have it as a dependency in your app/other modules, which you could still (in theory) compile with Jack.

[sbosley]

Hey @Aranda -- wanted to let you know I just re-tried the Jack compiler in the squidb-tests project with the latest stable android gradle plugin (2.2.0), and it seems to be working now! I had experimented with earlier versions of the plugin, and the build failures I saw made me suspicious that the cause was Jack bug in their implementation of the annotation processing API, but it seems like perhaps they've fixed it. Here was the configuration I used:

android {
    ...
    defaultConfig {
        jackOptions {
            enabled true
        }
        javaCompileOptions {
            annotationProcessorOptions {
                // Any arguments that would have gone in apt { arguments {...} } go here:
                arguments = [ squidbOptions : 'androidModels' ]
            } 
        }
    }
}
dependencies {
    ...
    annotationProcessor 'com.yahoo.squidb:squidb-processor:3.1.3'
}

I was able to get a clean build of our tests passing using this configuration. If you decide to give Jack a try again and it works for you with the latest android gradle plugin, let us know and I think we can close this issue.

[sbosley]

Hey @Aranda, did you ever get a chance to see if your issue was resolved? I'm inclined to close it -- I think the early bugs with the annotationProcessor configuration have been fixed, and we're now using it in place of the android-apt plugin in both the sample projects and our test suite, and I've tested things with Jack as well.

[AndroidDeveloperLB]

@sbosley Can you please tell if some gradle stuff needed to be removed ? On StackOverflow, they say to remove those:

classpath 'com.neenbedankt.gradle.plugins:android-apt:1.8'
apply plugin: 'com.neenbedankt.android-apt'
apt 'com.google.dagger:dagger-compiler:2.0'

and add this:

android{
...
        jackOptions {
            enabled true
        }
...
}

dependencies {
annotationProcessor 'com.google.dagger:dagger-compiler:2.0'
}

After doing so, I got this error:

Error:(232, 0) Could not find method apt() for arguments [com.yahoo.squidb:squidb-processor:3.2.0] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.

So I thought it needs the "apt" plugin, and I've re-added these:

classpath 'com.neenbedankt.gradle.plugins:android-apt:1.8'
apply plugin: 'com.neenbedankt.android-apt'

But now I get this error: Error:Could not get unknown property 'classpath' for task ':...' of type com.android.build.gradle.internal.pipeline.TransformTask.

[sbosley]

@AndroidDeveloperLB the android-apt plugin isn't compatible with the jack compiler. If you enable jack, you need to update your gradle file to completely remove the apt plugin, and use annotationProcessor everywhere. From the error message, it sounds like you missed one of these places and still had apt 'com.yahoo.squidb:squidb-processor:3.2.0' somewhere. android-apt is deprecated as of android gradle plugin 2.2.0; I recommend you follow the migration instructions as documented by android-apt, which can be found here.

Note that the android-apt plugin can be removed and replaced with the built-in annotationProcessor configuration as long as you are using android gradle plugin 2.2.0 or higher, regardless of whether or not you are using jack.

[Aranda]

Hi @sbosley

did you ever get a chance to see if your issue was resolved?

No, after disabling Jack our build times were reduced significantly, so we've not bothered to re-enable it yet. I'm happy for you to close this issue and we can reopen or create a new one if there are still problems in our particular setup. Thanks!

[sbosley]

Ok, sounds good! I noticed the same thing about build times when I was experimenting with Jack, so we haven't enabled it either. However, it does seem like the annotationProcessor configuration works to replace android-apt regardless of whether or not Jack is enabled, so you can feel free to at least update that part of it to future-proof things if you're so inclined -- not totally required though, so up to you.

[Aranda]

Thanks for the info re: annotationProcessor!

[AndroidDeveloperLB]

I'm still not sure how to enable it correctly, but as I see what you guys wrote now, you say it's probably not worth even trying Jack, as it makes the build time longer?

[sbosley]

@AndroidDeveloperLB that's right. There are actually two separate issues being discussed here:

1) Upgrading from the android-apt plugin to the built-in annotationProcessor configuration in gradle. This is something you can do regardless of whether or not you enable Jack. If you want to do this, you should follow the directions I linked to above. Any issues after having followed those instructions are likely due either to not fully removing android-apt (i.e. missing a step when following the instructions), or bugs in Android's gradle plugin. You can file issues with those projects if you have problems that are independent of SquiDB.
2) Enabling Jack. This does seem to increase build times so we're not using it, but SquiDB has been tested and confirmed to work with it if you do choose to enable it. If you do choose to enable Jack, doing the update to remove android-apt is required, but if you choose not to enable Jack, it's up to you whether or not to remove android-apt -- totally your choice.

[AndroidDeveloperLB]

@sbosley That's ok. If it makes build time slower, I won't even try it anytime soon. Don't need Java 1.8 this much. Our build time is very long already. Thank you for the good support. I hope that by the time Google releases a final version of the toolchain (or whatever it's called), it will be easy to move to it, and, hopefully, it will be worth the switch.

[AndroidDeveloperLB]

@sbosley Any news about this, and when it will be available? I wonder if Google also updated its builder.

[sbosley]

@AndroidDeveloperLB it should be working already; is it not for you? When we closed the issue was when it started working. As far as we've been able to tell this was never a SquiDB issue, but a build tools one. If you follow Google's instructions to enable jack and the android-apt instructions I linked to previously, things should be working fine.

[AndroidDeveloperLB]

@sbosley I mean Jack together with this library. Is it now working well, or is it still making the build time slower for this library ? As soon as you wrote that it makes the build time slower, I stopped even trying to use it. We already have a very long build time, no matter what we try. I was thinking of trying Jack, to both support Java 1.8 and make the build time shorter.

[sbosley]

@AndroidDeveloperLB right, I understand. We don't have any evidence to suggest that SquiDB specifically is making Jack slow -- I think Jack is just slow. Many people have reported similar issues, for example here and here.

We're still not using Jack, so I can't speak to whether or not Google has improved things. If you do find evidence that it's SquiDB specifically that is making Jack slow (this is highly unlikely as there shouldn't be anything special about SquiDB as compared to any other jar), please do share it!

[AndroidDeveloperLB]

@sbosley So we should wait for it to have a new version then.

[AndroidDeveloperLB]

Seems Google will not use Jack after all: https://android-developers.googleblog.com/2017/03/future-of-java-8-language-feature.html