search

square / moshi

A modern JSON library for Kotlin and Java.
https://square.github.io/moshi/1.x/
Apache License 2.0
9.76k stars 760 forks source link

Build exception when using new Proguard rules #738

Closed oleandre closed 5 years ago

oleandre commented 5 years ago

After updating to Moshi 1.7.0 and updating the Proguard-rules I've encountered a Gradle build exception:

Exception while processing task java.io.IOException: proguard.ParseException: Use of generics not allowed for java type at '<1><2><3>JsonAdapter' in line 31 of file '/Library/TeamCity/buildAgent/work/3ce9226b513c2493/dummy/proguard/proguard-moshi.pro'

I'm now using my old proguard rules, and the build works fine, but I would like to know how to use the new Proguard rules. Here's my current rules which seems to work fine:

-dontwarn okio. -dontwarn javax.annotation. -keepclassmembers class { @com.squareup.moshi.FromJson ; @com.squareup.moshi.ToJson ; } -keep @com.squareup.moshi.JsonQualifier interface -keepclassmembers class kotlin.Metadata { public ; } -dontwarn org.jetbrains.annotations.* -keep class kotlin.Metadata { ; }

-dontwarn kotlin.reflect.jvm.internal. -keep class kotlin.reflect.jvm.internal. { *; }

swankjesse commented 5 years ago

Java or Android? Does R8 work?

oleandre commented 5 years ago

Android. Haven't tried R8, should I?

swankjesse commented 5 years ago

Please do. We might have accidentally made use of an R8-only feature.

gabrielittner commented 5 years ago

From a quick test it seems like proguard doesn't like more than 2 parameters. Could be a parsing bug.

gabrielittner commented 5 years ago

It's a parsing bug that will be fixed in proguard 6.1.0-beta2 https://sourceforge.net/p/proguard/bugs/731/

ezamelczykespeo commented 5 years ago

Is it safe to omit these rules, or do we need to wait for the fix? Maybe some kind of workaround?

oleandre commented 5 years ago

I’m wondering if it’s safe to upgrade to 1.7.0 from 1.5.0, use the old rules and release app in production?

gabrielittner commented 5 years ago

It depends on how you use Moshi.

1) If you don't use the Kotlin moshi-kotlin-codegen artifact at all, it's save to just include this part of the file
https://github.com/square/moshi/blob/master/moshi/src/main/resources/META-INF/proguard/moshi.pro#L1-L14

2) If you do use moshi-kotlin-codegen it depends on whether you have nested classes that are annotated with @JsonClass. If you have no nesting or only one level of nesting you can use this part of the file
https://github.com/square/moshi/blob/master/moshi/src/main/resources/META-INF/proguard/moshi.pro#L1-L29

3) If you have deeper nesting you can workaround by using this:

# JSR 305 annotations are for embedding nullability information.
-dontwarn javax.annotation.**

-keepclasseswithmembers class * {
    @com.squareup.moshi.* <methods>;
}

-keep @com.squareup.moshi.JsonQualifier interface *

# Enum field names are used by the integrated EnumJsonAdapter.
# Annotate enums with @JsonClass(generateAdapter = false) to use them with Moshi.
-keepclassmembers @com.squareup.moshi.JsonClass class * extends java.lang.Enum {
    <fields>;
}

# The name of @JsonClass types is used to look up the generated adapter.
-keepnames @com.squareup.moshi.JsonClass class *

# Retain generated JsonAdapters if annotated type is retained.
-keep class **JsonAdapter {
    <init>(...);
    <fields>;
}

(this replaces the more specific keep rules with one that keeps all classes with a name that ends with JsonAdapter)

cdongieux commented 5 years ago

Hi,

Do you plan to fix rules in the lib's moshi.pro file for ProGuard users? I currently can't use R8 and as the moshi.pro is automatically parsed, I have to disable minifying for my project to build.

Best regards

swankjesse commented 5 years ago

Any idea how we’d fix it for ProGuard users?

cdongieux commented 5 years ago

I guess what @gabrielittner provided should be fine: https://github.com/square/moshi/issues/738#issuecomment-437281870

julioromano commented 5 years ago

We're using moshi-kotlin-codegen 1.8.0 without any issues since it's been released (19 days ago) but have incurred into this build error yesterday after upgrading Android Gradle plugin from 3.2.1 to 3.3.0-rc01.

No proguard config was changed in the process.

Is this expected behavior?

PaulWoitaschek commented 5 years ago

Same here with 3.3.0-rc01. How can this be fixed?

mbernier85 commented 5 years ago

Could it be related to this

https://sourceforge.net/p/proguard/bugs/731/

technoir42 commented 5 years ago

You can force Gradle to use a newer version of ProGuard:

buildscript {
    configurations.all {
        resolutionStrategy {
            force 'net.sf.proguard:proguard-gradle:6.1.0beta1'
        }
    }
}

However version 6.1.0beta2 that should contain a fix unfortunately hasn't been released yet.

technoir42 commented 5 years ago

Just found this flag in the AGP source code:

android.proguard.enableRulesExtraction=false

Works for 3.3.0-rc02, but you'll need to add the rules manually without the part which confuses ProGuard. You don't need this if you've enabled R8 or use 3.4.0 where it's enabled by default.

gabrielittner commented 5 years ago

Proguard 6.1.0beta2 is out. Here is a summary of the workarounds:

AGP 3.2.x

Copy the proguard rules to your project like described in this comment.

AGP 3.3.x

Set android.proguard.enableRulesExtraction=false and then copy the proguard rules to your project like described in this comment.

OR

Upgrade proguard to 6.1.0beta2:

buildscript {
    configurations.all {
        resolutionStrategy {
            force 'net.sf.proguard:proguard-gradle:6.1.0beta2'
        }
    }
}

OR

Enable R8 by setting android.enableR8=true.

AGP 3.4.x

Should work by default. If you disabled R8 follow the instructions for AGP 3.3.

cdongieux commented 5 years ago

Great! Thanks a lot @gabrielittner!

svenjacobs commented 5 years ago

Moshi: 1.8.0 Android Gradle Plugin: 3.3.0

I enabled R8 via android.enableR8=true but still receive this error :(

Update:

I disabled rules extraction via android.proguard.enableRulesExtraction=false and copied the Moshi rules into my own ProGuard file. Now I receive the same error regarding <1>_<2>_<3>JsonAdapter in my own file. I'm not sure why ProGuard still tries to parse these files although R8 is enabled?!

Update 2:

Disabling R8 again and forcing ProGuard 6.1.0beta2 now gives me the following error:

> Failed to notify project evaluation listener.
   > proguard.ConfigurationParser.parseClassSpecificationArguments()Lproguard/ClassSpecification;

I seem to be stuck :(

Update 3:

Explicitly specifying useProguard false wherever minifyEnabled true is set did also not help.

gabrielittner commented 5 years ago

@svenjacobs Do you have a sample project? Both R8 and ProGuard 6.1.0beta2 work for me.

In regards to your first update. You shouldn't copy all of the rules, only parts of it like described here https://github.com/square/moshi/issues/738#issuecomment-437281870

svenjacobs commented 5 years ago

@gabrielittner I don't have a sample project as the project where the error occurs is quite complex and consists of several modules. I would have to create a sample project.

svenjacobs commented 5 years ago

@gabrielittner I found the issue. The project consists of multiple feature modules. Each feature contained the ProGuard configuration per build type. Although the configuration was identical, once I removed the configuration from the feature modules and only kept it for the app module everything works.

yogurtearl commented 5 years ago

As a heads up, the Fabric gradle plugin doesn't work with latest beta of ProGuard. It gives this error during configuration. Fabric triggers the issue, but I don't see Fabric in the stacktrace, so it maybe a problem in AGP. 

Caused by: java.lang.NoSuchMethodError: proguard.ConfigurationParser.parseClassSpecificationArguments()Lproguard/ClassSpecification;
        at com.android.build.gradle.internal.transforms.BaseProguardAction.keep(BaseProguardAction.java:79)
        at com.android.build.gradle.internal.TaskManager.applyProguardDefaultsForTest(TaskManager.java:3433)
        at com.android.build.gradle.internal.TaskManager.applyProguardRules(TaskManager.java:3366)
        at com.android.build.gradle.internal.TaskManager.createProguardTransform(TaskManager.java:3341)
        at com.android.build.gradle.internal.TaskManager.doCreateJavaCodeShrinkerTransform(TaskManager.java:3279)
        at com.android.build.gradle.internal.TaskManager.maybeCreateJavaCodeShrinkerTransform(TaskManager.java:3244)
        at com.android.build.gradle.internal.TaskManager.createPostCompilationTasks(TaskManager.java:2181)
        at com.android.build.gradle.internal.TaskManager.createAndroidTestVariantTasks(TaskManager.java:1808)
        at com.android.build.gradle.internal.VariantManager.createTasksForVariantData(VariantManager.java:485)
        at com.android.build.gradle.internal.VariantManager.createAndroidTasks(VariantManager.java:365)
        at com.android.build.gradle.BasePlugin.createAndroidTasks(BasePlugin.java:767)
        at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:81)
        at com.android.build.gradle.BasePlugin.lambda$createTasks$4(BasePlugin.java:651)
        at com.android.build.gradle.internal.crash.CrashReporting$afterEvaluate$1.execute(crash_reporting.kt:37)
        at com.android.build.gradle.internal.crash.CrashReporting$afterEvaluate$1.execute(crash_reporting.kt)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1$1.run(DefaultListenerBuildOperationDecorator.java:150)
        at org.gradle.configuration.internal.DefaultUserCodeApplicationContext.reapply(DefaultUserCodeApplicationContext.java:58)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1.run(DefaultListenerBuildOperationDecorator.java:147)
mrwillis21 commented 5 years ago

This doesn't appear to be specific to Fabric. I get the same error in a non-Fabric app when simply running ./gradlew clean with AGP 3.3.0 and Proguard 6.1.0beta2. Reverting to the default version of Proguard fixes this error.

Caused by: java.lang.NoSuchMethodError: proguard.ConfigurationParser.parseClassSpecificationArguments()Lproguard/ClassSpecification;
        at com.android.build.gradle.internal.transforms.BaseProguardAction.keep(BaseProguardAction.java:79)
        at com.android.build.gradle.internal.TaskManager.applyProguardDefaultsForTest(TaskManager.java:3433)
        at com.android.build.gradle.internal.TaskManager.applyProguardRules(TaskManager.java:3366)
        at com.android.build.gradle.internal.TaskManager.createProguardTransform(TaskManager.java:3341)
        at com.android.build.gradle.internal.TaskManager.doCreateJavaCodeShrinkerTransform(TaskManager.java:3279)
        at com.android.build.gradle.internal.TaskManager.maybeCreateJavaCodeShrinkerTransform(TaskManager.java:3244)
        at com.android.build.gradle.internal.TaskManager.createPostCompilationTasks(TaskManager.java:2181)
        at com.android.build.gradle.internal.TaskManager.createAndroidTestVariantTasks(TaskManager.java:1808)
        at com.android.build.gradle.internal.VariantManager.createTasksForVariantData(VariantManager.java:485)
        at com.android.build.gradle.internal.VariantManager.createAndroidTasks(VariantManager.java:365)
        at com.android.build.gradle.BasePlugin.createAndroidTasks(BasePlugin.java:767)
        at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:81)
        at com.android.build.gradle.BasePlugin.lambda$createTasks$4(BasePlugin.java:651)
        at com.android.build.gradle.internal.crash.CrashReporting$afterEvaluate$1.execute(crash_reporting.kt:37)
        at com.android.build.gradle.internal.crash.CrashReporting$afterEvaluate$1.execute(crash_reporting.kt)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1$1.run(DefaultListenerBuildOperationDecorator.java:157)
        at org.gradle.configuration.internal.DefaultUserCodeApplicationContext.reapply(DefaultUserCodeApplicationContext.java:58)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1.run(DefaultListenerBuildOperationDecorator.java:154)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:301)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:293)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction.execute(DefaultListenerBuildOperationDecorator.java:151)
        at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:91)
        at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:80)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
        ... 110 more
pawelkw commented 5 years ago

Try this rule as a workaround:


-keep public class * extends com.squareup.moshi.JsonAdapter {
    <init>(...);
    <fields>;
}

And remove all the other rules that include <1>_<2> and so one (there's like 6 of them). Working for me without any other changes. Make sure the included rules with the moshi dependency are not in use. I use moshi 1.6.0, so they're not there AFAIK, but you get them if you use 1.8.0.

PS: This is for codegen only! PS2: Copy-pasta rules here: https://gist.github.com/pawelkw/c5bc377ea4fc6cdbd93bc770c0895879

minimoz commented 5 years ago

I had the exact same issue with AGP 3.3.0, Proguard 6.1.0beta2 and moshi 1.8.0 in my custom libraries. My workaround was to ignore proguard rules being extracted by adding the following to gradle.properties file :

android.proguard.enableRulesExtraction=false

⚠️ this option setting is experimental

JulienGenoud commented 5 years ago

i had this issue when i updated :

I just rollback this and everything is working correctly.

Antimonit commented 5 years ago

@gabrielittner I found the issue. The project consists of multiple feature modules. Each feature contained the ProGuard configuration per build type. Although the configuration was identical, once I removed the configuration from the feature modules and only kept it for the app module everything works.

This was our case too. Only the main :app module should define proguardFiles(). If you want specific proguard rules for your feature modules, you can define additional rules using consumerProguardFiles():

buildTypes {
    release {
        consumerProguardFiles("proguard-rules.pro")
    }
}
HovadoET commented 5 years ago

Nice

skzwksk commented 5 years ago

I faced the same problem when updating android studio 3.4.1

ZacSweers commented 5 years ago

Closing this out now as R8 is now the default in the android gradle plugin and Proguard 6.1.0 stable (and 6.1.1!) is released with support for the modern rules. Please use one of these versions going forward

marcoRS commented 5 years ago

How would we specify to use Proguard 6.1.0+ ? I am using on moshi version 1.8.0 and Android Plugin 3.3.2

changety commented 5 years ago

Try this rule as a workaround:


-keep public class * extends com.squareup.moshi.JsonAdapter {
    <init>(...);
    <fields>;
}

And remove all the other rules that include <1>_<2> and so one (there's like 6 of them). Working for me without any other changes. Make sure the included rules with the moshi dependency are not in use. I use moshi 1.6.0, so they're not there AFAIK, but you get them if you use 1.8.0.

PS: This is for codegen only! PS2: Copy-pasta rules here: https://gist.github.com/pawelkw/c5bc377ea4fc6cdbd93bc770c0895879

how to Make sure the included rules with the moshi dependency are NOT IN USE?

Drjacky commented 4 years ago

Android Studio 3.5.2 Build #AI-191.8026.42.35.5977832, built on October 30, 2019 JRE: 1.8.0_202-release-1483-b49-5587405 x86_64 JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o macOS 10.15.1 Kotlin 1.3.60

  • What went wrong: Execution failed for task ':app:transformClassesAndResourcesWithProguardForAlpha'. java.io.IOException: proguard.ParseException: Use of generics not allowed for java type at '<1><2><3>JsonAdapter' in line 43 of file '/Users/drjacky/.gradle/caches/transforms-2/files-2.1/4d48b9430371242423f570ca1beef7fc/META-INF/proguard/moshi.pro'

Proguard:

-printconfiguration "build/outputs/mapping/configuration.txt"

-keepattributes Signature
-keepattributes Exceptions
-keepattributes *Annotation*
-keepattributes SourceFile,LineNumberTable

-dontwarn okhttp3.**
-dontwarn retrofit2.Platform
-dontwarn retrofit2.Platform$Java8
-dontwarn okio.**
-dontwarn javax.annotation.**
-dontwarn org.jetbrains.annotations.**
-dontwarn com.google.errorprone.annotations.*
-dontwarn java.lang.invoke.*
-dontwarn org.codehaus.mojo.animal_sniffer.*

-keep class retrofit2.** { *; }
-keep class okio.** { *; }
-keep class okhttp3.** { *; }
-keep class io.reactivex.** { *; }
-keep class org.reactivestreams.** { *; }
-keep class kotlin.Metadata { *; }
-keep @org.parceler.Parcel class * { *; }
-keep class **$$Parcelable { *; }
-keep interface org.parceler.Parcel
-keep public class * extends java.lang.Exception
-keep @com.squareup.moshi.JsonQualifier interface *
-keep interface kotlin.reflect.jvm.internal.impl.builtins.BuiltInsLoader
-keep class kotlin.reflect.jvm.internal.impl.builtins.BuiltInsLoaderImpl
-keep class kotlin.reflect.jvm.internal.impl.serialization.deserialization.builtins.BuiltInsLoaderImpl
-keep public class * implements com.bumptech.glide.module.GlideModule
-keep public class * extends com.bumptech.glide.module.AppGlideModule
-keep public enum com.bumptech.glide.load.ImageHeaderParser$** {
    **[] $VALUES;
    public *;
}

-keepclasseswithmembers class * {
    @retrofit2.http.* <methods>;
}
-keepclasseswithmembers class * {
    @com.squareup.moshi.* <methods>;
}

-keepclassmembers class kotlin.Metadata {
    public <methods>;
}
-keepclassmembers class * {
    @com.squareup.moshi.FromJson <methods>;
    @com.squareup.moshi.ToJson <methods>;
}

# Enum field names are used by the integrated EnumJsonAdapter.
# Annotate enums with @JsonClass(generateAdapter = false) to use them with Moshi.
-keepclassmembers @com.squareup.moshi.JsonClass class * extends java.lang.Enum {
    <fields>;
}

-keepclassmembers class kotlin.Metadata {
    public <methods>;
}

# The name of @JsonClass types is used to look up the generated adapter.
-keepnames @com.squareup.moshi.JsonClass class *

# Retain generated JsonAdapters if annotated type is retained.
#Uncomment when this bug gets solved: https://youtrack.jetbrains.com/issue/KT-29668
#-if @com.squareup.moshi.JsonClass class *
#-keep class <1>JsonAdapter {
#    <init>(...);
#    <fields>;
#}
#-if @com.squareup.moshi.JsonClass class **$*
#-keep class <1>_<2>JsonAdapter {
#    <init>(...);
#    <fields>;
#}
#-if @com.squareup.moshi.JsonClass class **$*$*
#-keep class <1>_<2>_<3>JsonAdapter {
#    <init>(...);
#    <fields>;
#}
#-if @com.squareup.moshi.JsonClass class **$*$*$*
#-keep class <1>_<2>_<3>_<4>JsonAdapter {
#    <init>(...);
#    <fields>;
#}
#-if @com.squareup.moshi.JsonClass class **$*$*$*$*
#-keep class <1>_<2>_<3>_<4>_<5>JsonAdapter {
#    <init>(...);
#    <fields>;
#}
#-if @com.squareup.moshi.JsonClass class **$*$*$*$*$*
#-keep class <1>_<2>_<3>_<4>_<5>_<6>JsonAdapter {
#    <init>(...);
#    <fields>;
#}

# Moshi - Keep entity names
-keepnames @kotlin.Metadata class com.safeboda.data.entity.**
-keep class com.safeboda.data.entity.** { *; }
-keepclassmembers class com.safeboda.data.entity.** { *; }
-keepnames @kotlin.Metadata class com.safeboda.domain.entity.**
-keep class com.safeboda.domain.entity.** { *; }
-keepclassmembers class com.safeboda.domain.entity.** { *; }
ZacSweers commented 4 years ago

@drjacky see my comment in #1042

Drjacky commented 4 years ago

@ZacSweers Please take a look at https://github.com/square/moshi/issues/1042#issuecomment-558718625

owayne1 commented 1 year ago

how to fix this error in Gradle 7.4 up to Gradle Version 7.5.1 please help i try everything

Expecting type and name instead of just 'setExtensionCallback' before '(' in line 380 of file '/Users/AndroidStudioProjects/ProjeckfileName/app/build/intermediates/proguard/configs/release/consumer-rules.pro'