First initialization jacksonObjectMapper() and first parse of .readValue is quite slow

I have issue with Android 4.1.X and Kotlin version 1.1.X first initialization ObjectMapper and first parse of readValue method is quite slow. I did comparison with latest version of Kotson (Gson for Kotlin) and see the difference.

Here is the test code:

import android.support.v7.app.AppCompatActivity
import android.os.Bundle
import com.fasterxml.jackson.annotation.JsonProperty
import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
import com.github.salomonbrys.kotson.fromJson
import com.google.gson.Gson
import com.google.gson.annotations.SerializedName
import kotlinx.android.synthetic.main.activity_test_json_parse.tv_hello_world

class TestJsonParseActivity : AppCompatActivity() {

    val json: String = """[{"all_total_items":12,"all_remain_items":12,"all_used_items":0,"total_items":1,"remain_items":1,"used_items":0,"package_id":47,"package_sku":"FootMassagePack","package_name":"Foot Massage 12 S","package_sessions":[{"total_items":12,"remain_items":12,"used_items":0,"session_id":118,"session_sku":"FootMassage","session_name":"Foot Massage 12 sessions"}],"transaction_number":"FMT00000085"},{"all_total_items":21,"all_remain_items":18,"all_used_items":3,"total_items":1,"remain_items":1,"used_items":0,"package_id":46,"package_sku":"SG50-PACKAGE","package_name":"Singapore Hair Salon Anniversary Pack","package_sessions":[{"total_items":5,"remain_items":4,"used_items":1,"session_id":115,"session_sku":"FINGERNAILCOLOR","session_name":"Finger nails color"},{"total_items":1,"remain_items":0,"used_items":1,"session_id":116,"session_sku":"1HRFULLMASSAGE","session_name":"1 Hour Full Body Massage"},{"total_items":15,"remain_items":14,"used_items":1,"session_id":117,"session_sku":"HAIRTREATMENT01","session_name":"Special Hair Treatment"}],"transaction_number":"FMT00000081"}]"""

    data class ChildSession (

        @JsonProperty("total_items")
        @SerializedName("total_items")
        val totalItems: Int,

        @JsonProperty("remain_items")
        @SerializedName("remain_items")
        val remainItems: Int,

        @JsonProperty("used_items")
        @SerializedName("used_items")
        val usedItems: Int,

        @JsonProperty("session_id")
        @SerializedName("session_id")
        val sessionId: Int,

        @JsonProperty("session_sku")
        @SerializedName("session_sku")
        val sessionSku: String,

        @JsonProperty("session_name")
        @SerializedName("session_name")
        val sessionName: String
        )

    data class Package (

            @JsonProperty("all_total_items")
            @SerializedName("all_total_items")
            val allTotalItems: Int,

            @JsonProperty("all_remain_items")
            @SerializedName("all_remain_items")
            val allRemainItems: Int,

            @JsonProperty("all_used_items")
            @SerializedName("all_used_items")
            val allUsedItems: Int,

            @JsonProperty("total_items")
            @SerializedName("total_items")
            val totalItems: Int,

            @JsonProperty("remain_items")
            @SerializedName("remain_items")
            val remainItems: Int,

            @JsonProperty("used_items")
            @SerializedName("used_items")
            val usedItems: Int,

            @JsonProperty("package_id")
            @SerializedName("package_id")
            val packageId: Int,

            @JsonProperty("package_sku")
            @SerializedName("package_sku")
            val packageSku: String,

            @JsonProperty("package_name")
            @SerializedName("package_name")
            val packageName: String,

            @JsonProperty("transaction_number")
            @SerializedName("transaction_number")
            val transactionNumber: String,

            @JsonProperty("package_sessions")
            @SerializedName("package_sessions")
            val sessions: List<ChildSession>
    )

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_test_json_parse)

        val init_time = System.currentTimeMillis()
        //val gson = Gson()
        val jackson = jacksonObjectMapper()
        //val packages: List<Package> = gson.fromJson(json)
        val packages: List<Package> = jackson.readValue(json)
        val execution_time = System.currentTimeMillis() - init_time
        tv_hello_world.text = "Execution_time: " + execution_time + ", Result: " + packages.toString()
    }
}

Here is my gradle file:

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
apply plugin: 'kotlin-kapt'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.2"
    defaultConfig {
        applicationId "json.parse.test.com.testjsonparse"
        minSdkVersion 16
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }

    packagingOptions {
        pickFirst 'META-INF/DEPENDENCIES.txt'
        pickFirst 'META-INF/LICENSE.txt'
        pickFirst 'META-INF/NOTICE.txt'
        pickFirst 'META-INF/NOTICE'
        pickFirst 'META-INF/LICENSE'
        pickFirst 'META-INF/DEPENDENCIES'
        pickFirst 'META-INF/notice.txt'
        pickFirst 'META-INF/license.txt'
        pickFirst 'META-INF/dependencies.txt'
        pickFirst 'META-INF/LGPL2.1'
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.3.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.2'
    testCompile 'junit:junit:4.12'
    compile "org.jetbrains.kotlin:kotlin-stdlib-jre7:$kotlin_version"
    compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
    compile 'com.github.salomonbrys.kotson:kotson:2.5.0'
    compile "com.fasterxml.jackson.module:jackson-module-kotlin:2.8.7"
}
repositories {
    mavenCentral()
}

So execution time on emulator:

Jackson 454 milliseconds
Gson 4 milliseconds

Well still acceptable but when I run same code on hardware device itself the result is dramatic:

Jackson 1759 milliseconds
Gson 32 milliseconds

Almost 2 seconds for first initialization and parse. Why? Then after initialization it work quite fast. But form time to time (especially if app consume a lot of memory) this few seconds delay appear again because Android re-initialize ObjectMapper. In production is totally unacceptable. I have to switch form your library to Gson. Is it possible to fix it and reduce time for initialization?

You are somehow assuming that this is a bug in Jackson, but unfortunately this is Android "feature": access to annotations is hideously slow, and there is no way around that part as far as I know. For some reason Google does not consider it a problem.

Since Jackson allows extensive configuration via annotations it has to introspect annotations for classes, constructors, fields and methods: but after (de)serializer for given type has been resolved no further access is needed. I think Gson does not allow as extensive configuration, but it also does not necessarily cache handlers on first access so it will likely incur higher overhead for further calls. That is a trade-off.

Now: if you do not use annotations for configuration (but rely on visibility rules, naming convention), you may disable annotation use with

 mapper.disable(MapperFeature.USE_ANNOTATIONS);

and that should reduce first-time impact significantly.

Another possibility is to try to do first call as early as possible (ideally immediately on app startup), before actual need. While this does not speed up initialization, it will incur the cost earlier and possibly help end-to-end initialization time.

As to recurring cost: Jackson itself retains resolved (de)serializers so there is nothing that should lead to increased cost. But Android is quite aggressive in its efforts to push out various components, so this may be a side-effect of some kind. As with most other aspects I do not know what Jackson could do to alleviate this -- I am open to suggestions, or perhaps links to articles if anyone is familiar with related issues.

Besides standard Jackson and GSON, another library that might work better for some Android cases is:

https://github.com/FasterXML/jackson-jr

which is based on jackson-core streaming API, which is very high-performance on platforms including Android. But it does not use annotations for anything, so whenever it works well enough for use case (with limited feature set) its initialization overhead is rather low. But sustained performance comparable to standard Jackson.

@cowtowncoder if I use mapper.disable(MapperFeature.USE_ANNOTATIONS);

I getting this exception

A resource was acquired at attached stack trace but never released. See java.io.Closeable for information on avoiding resource leaks.
                                                                    java.lang.Throwable: Explicit termination method 'end' not called
                                                                        at dalvik.system.CloseGuard.open(CloseGuard.java:184)
                                                                        at java.util.zip.Inflater.<init>(Inflater.java:82)
                                                                        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:96)
                                                                        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:81)
                                                                        at libcore.net.http.HttpEngine.initContentStream(HttpEngine.java:528)
                                                                        at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:836)
                                                                        at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:274)
                                                                        at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:486)
                                                                        at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)
                                                                        at com.google.android.gms.http.GoogleHttpClient.a(:com.google.android.gms:797)
                                                                        at com.google.android.gms.http.GoogleHttpClient.a(:com.google.android.gms:762)
                                                                        at com.google.android.gms.http.GoogleHttpClient.execute(:com.google.android.gms:669)
                                                                        at com.google.android.gms.http.GoogleHttpClient.execute(:com.google.android.gms:653)
                                                                        at cyh.a(:com.google.android.gms:233)
                                                                        at dmx.a(:com.google.android.gms:263)
                                                                        at dmx.a(:com.google.android.gms:4235)
                                                                        at dmw.a(:com.google.android.gms:47)
                                                                        at dmq.a(:com.google.android.gms:55)
                                                                        at dmp.a(:com.google.android.gms:113)
                                                                        at com.google.android.gms.auth.account.be.legacy.AuthCronChimeraService.a(:com.google.android.gms:3054)
                                                                        at mwx.run(:com.google.android.gms:179)

@atonamy That does not seem to have any Jackson-related classes in it? I don't know how lack of annotation processing should change handling of input/output in anyway; it is quite separate functionality. So unfortunately I don't know what would be happening here.

@cowtowncoder Sorry wrong exception. I mean this one: FATAL EXCEPTION: main

java.lang.RuntimeException: Unable to start activity ComponentInfo{json.parse.test.com.testjsonparse/json.parse.test.com.testjsonparse.TestJsonParseActivity}: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of json.parse.test.com.testjsonparse.TestJsonParseActivity$Package: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)
                                                                                     at [Source: [{"all_total_items":12,"all_remain_items":12,"all_used_items":0,"total_items":1,"remain_items":1,"used_items":0,"package_id":47,"package_sku":"FootMassagePack","package_name":"Foot Massage 12 S","package_sessions":[{"total_items":12,"remain_items":12,"used_items":0,"session_id":118,"session_sku":"FootMassage","session_name":"Foot Massage 12 sessions"}],"transaction_number":"FMT00000085"},{"all_total_items":21,"all_remain_items":18,"all_used_items":3,"total_items":1,"remain_items":1,"used_items":0,"package_id":46,"package_sku":"SG50-PACKAGE","package_name":"Singapore Hair Salon Anniversary Pack","package_sessions":[{"total_items":5,"remain_items":4,"used_items":1,"session_id":115,"session_sku":"FINGERNAILCOLOR","session_name":"Finger nails color"},{"total_items":1,"remain_items":0,"used_items":1,"session_id":116,"session_sku":"1HRFULLMASSAGE","session_name":"1 Hour Full Body Massage"},{"total_items":15,"remain_items":14,"used_items":1,"session_id":117,"session_sku":"HAIRTREATMENT01","session_name":"Special Hair Treatment"}],"transaction_number":"FMT00000081"}]; line: 1, column: 3] (through reference chain: java.util.ArrayList[0])
                                                                                     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2059)
                                                                                     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2084)
                                                                                     at android.app.ActivityThread.access$600(ActivityThread.java:130)
                                                                                     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1195)
                                                                                     at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                                     at android.os.Looper.loop(Looper.java:137)
                                                                                     at android.app.ActivityThread.main(ActivityThread.java:4745)
                                                                                     at java.lang.reflect.Method.invokeNative(Native Method)
                                                                                     at java.lang.reflect.Method.invoke(Method.java:511)
                                                                                     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
                                                                                     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
                                                                                     at dalvik.system.NativeStart.main(Native Method)
                                                                                  Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of json.parse.test.com.testjsonparse.TestJsonParseActivity$Package: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)
                                                                                     at [Source: [{"all_total_items":12,"all_remain_items":12,"all_used_items":0,"total_items":1,"remain_items":1,"used_items":0,"package_id":47,"package_sku":"FootMassagePack","package_name":"Foot Massage 12 S","package_sessions":[{"total_items":12,"remain_items":12,"used_items":0,"session_id":118,"session_sku":"FootMassage","session_name":"Foot Massage 12 sessions"}],"transaction_number":"FMT00000085"},{"all_total_items":21,"all_remain_items":18,"all_used_items":3,"total_items":1,"remain_items":1,"used_items":0,"package_id":46,"package_sku":"SG50-PACKAGE","package_name":"Singapore Hair Salon Anniversary Pack","package_sessions":[{"total_items":5,"remain_items":4,"used_items":1,"session_id":115,"session_sku":"FINGERNAILCOLOR","session_name":"Finger nails color"},{"total_items":1,"remain_items":0,"used_items":1,"session_id":116,"session_sku":"1HRFULLMASSAGE","session_name":"1 Hour Full Body Massage"},{"total_items":15,"remain_items":14,"used_items":1,"session_id":117,"session_sku":"HAIRTREATMENT01","session_name":"Special Hair Treatment"}],"transaction_number":"FMT00000081"}]; line: 1, column: 3] (through reference chain: java.util.ArrayList[0])
                                                                                     at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1456)
                                                                                    at com.fasterxml.jackson.databind.Deserialization

It means without annotation I have to write my own custom deserializer for each class?

I just was thinking is it possible in future to provide optional normal annotations feature (same way as Gson support simple annotations). So for me who no need extensive configuration use this feature (and speed up first initialization) then if later I need more flexibility I just switch to extensive one. If this option has enough demand of course.

@atonamy No, without annotations you should be able to use default POJO handling, in general. But you will need to follow naming conventions to avoid having to use annotation-dictated overrides or changes.

As exception points out, the class in question does not have a 0-argument constructor to use. If you did rely on, say, @JsonCreator (to use alternative constructor, or maybe static factory method), then this would make sense. But It all depends on which annotations you were relying on, if any. It is unfortunately difficult say anything definite without knowing classes you are trying to deserialize here.

Oh. Alternatively... it may be that Kotlin module requires use of annotation introspector. If so my suggestion is irrelevant here. @apatrida can probably comment on this.

The kotlin plugin basically infers the JsonCreator annotation on behalf of databind. If it doesn't see one already existing, then when asked about it by the databind engine it lies a little bit and says which constructor it wants to be considered JsonCreator. Maybe your setting change there turns off this behavior, therefore the plugin no longer has the opportunity to manage the constructor selection or to use special Kotlin behaviors for doing so. (that's my guess as to why it falls apart with that setting turned on) I'm not sure how to solve it unless there is another plugin point for databind that allows for controlling construction ... then again the plugin itself looks for annotations so it might cause some slowness again when it does so.

Ah. That would explain it. When disabling use of annotations, call to AnnotationIntrospector is essentially blocked (well, there's a bogus introspector).

So much for that suggestion I guess, at least wrt current versions.

I also encountered somewhat similar problem while trying to migrate from AutoValue to Kotlin data classes. Also on Android.

Replacing single AutoValue class that uses builder for deserialization

@AutoValue
@JsonDeserialize(builder = AutoValue_Category.Builder.class)
public abstract class Category implements Serializable {
    @JsonProperty("id")
    public abstract int id();
    @NonNull
    @JsonProperty("name")
    public abstract String name();
    @NonNull
    @JsonProperty("human_url")
    public abstract String humanUrl();

    @NonNull
    public static Builder builder() {
        return new AutoValue_Category.Builder();
    }
    @NonNull
    public abstract Builder toBuilder();

    @AutoValue.Builder
    public abstract static class Builder {
        @JsonProperty("id")
        public abstract Builder id(int id);
        @JsonProperty("name")
        public abstract Builder name(@NonNull String name);
        @JsonProperty("human_url")
        public abstract Builder humanUrl(@NonNull String humanUrl);

        @NonNull
        public abstract Category build();
    }
}

with equivalent Kotlin data class

data class Category(@JsonProperty("id") val id: Int,
                    @JsonProperty("name") val name: String,
                    @JsonProperty("human_url") val humanUrl: String) : Serializable

results in dramatic slowdown of initial response parsing from ~47 ms to ~1800 ms. This makes combination of Jackson and Kotlin practically unusable on Android.

Is it possible to make deserialization of Kotlin classes somewhat similar in terms of performance to Java classes? Or is it impossible due to reflection usage and known problems with reflection on Android?

I attached part of tracing information from Android Studio, maybe it can be helpful. jackson_kotlin_1 jackson_kotlin_2

I'm seeing similar slow deserialization times.

I recently found a similar issue on a slow older device.

My suggestion, although I am not sure if it is possible would be to have an option to do compile time processing of annotations and generate classes to try to make the runtime process overhead smaller, again not really sure if it makes sense considering what Jackson internals do.

Is this still an issue on Android 4.4 plus, which replaced Dalvik with Android Runtime?

Closed unless further reports. Note that even on fully-fledged JVMs initializing Jackson is time consuming due to the class loading involved. A simple workaround is to create an ObjectMapper at application startup in a background thread:

Thread { ObjectMapper() }.start()

You don't even have to keep that reference around—merely starting that thread will do all of the (slow) classloading.

Seconded. I would suggest that even to load most Jackson classes, it is probably best to exercise simple read/write operations on background load (just read from simplest String to a simple value class; similarly write as String, discard).

And from that, introspection on Android has been historically unexpectedly inefficient and slow, so a big speedup might come from both initializing and keeping a pre-loaded instance. It is worth noting that if doing this, it is important to handle instantiation (construction plus configuration of mapper itself) in a thread-safe manner; but "warm-up" operation need not be synchronized (read and write operations on configured mapper are fully thread-safe; initial configuration isn't).

FasterXML / jackson-module-kotlin

First initialization jacksonObjectMapper() and first parse of .readValue is quite slow #69