Duplicated `@Deprecated` annotations when extending Java interface with deprecated default method cause `java.lang.annotation.AnnotationFormatError` when accessed via Java reflection (2.13.11 regression)

DieBauer commented 1 year ago

Reproduction steps

Scala version: 2.13.11

Reproducer is here: run sbt test showcases the behavior. Changing scala version to 2.13.10 solves it.

https://github.com/DieBauer/scala-reproducer/tree/master

package example;

interface A {
  @Deprecated
    default String a() {
        return "a";
    }
}

this is compiled in a separate stage (module)

and the scala class is

package example

class B extends A

then checking that the Deprecated annotation is present on the Scala class/method fails

Method deprecatedMethod = B.class.getMethod("a");
deprecatedMethod.isAnnotationPresent(Deprecated.class);

This fails with java.lang.annotation.AnnotationFormatError: Duplicate annotation for class: interface java.lang.Deprecated: @java.lang.Deprecated(forRemoval=false, since="")

possibly introduced by https://github.com/scala/scala/pull/10291

Problem

This issue is reproduced on Java 8, 11 and 17 (potentially higher versions as well). On 2.13.10 it does not occur.

When a Java class has a Deprecated method that is inherited in a Scala class, then checking the method through the scala class for present annotations throws a java.lang.annotation.AnnotationFormatError: Duplicate annotation for class: interface java.lang.Deprecated: @java.lang.Deprecated(forRemoval=false, since="")

[error]     at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:126)
[error]     at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:73)
[error]     at java.lang.reflect.Executable.declaredAnnotations(Executable.java:604)
[error]     at java.lang.reflect.Executable.declaredAnnotations(Executable.java:602)
[error]     at java.lang.reflect.Executable.getAnnotation(Executable.java:572)
[error]     at java.lang.reflect.Method.getAnnotation(Method.java:695)
[error]     at java.lang.reflect.AnnotatedElement.isAnnotationPresent(AnnotatedElement.java:274)
[error]     at java.lang.reflect.AccessibleObject.isAnnotationPresent(AccessibleObject.java:517)
[error]     at example.ExampleTest.java(ExampleTest.java:10)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.lang.reflect.Method.invoke(Method.java:566)

The issue is the duplicate Deprecated annotation in the bytecode when compiled with Scala 2.13.11

    RuntimeVisibleAnnotations:
      0: #13()
        java.lang.Deprecated
      1: #13()
        java.lang.Deprecated

Details

The java class bytecode looks like:

  Last modified 6 Jun 2023; size 277 bytes
  SHA-256 checksum 3b809a1b555c263005052518a2f9c1b04188c5c7a42c4a58761e0bc6eaef6863
  Compiled from "A.java"
public interface A
  minor version: 0
  major version: 55
  flags: (0x0601) ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT
  this_class: #1                          // A
  super_class: #2                         // java/lang/Object
  interfaces: 0, fields: 0, methods: 1, attributes: 1
Constant pool:
   #1 = Class              #12            // A
   #2 = Class              #13            // java/lang/Object
   #3 = Utf8               a
   #4 = Utf8               (Ljava/lang/String;)Ljava/lang/String;
   #5 = Utf8               Code
   #6 = Utf8               LineNumberTable
   #7 = Utf8               Deprecated
   #8 = Utf8               RuntimeVisibleAnnotations
   #9 = Utf8               Ljava/lang/Deprecated;
  #10 = Utf8               SourceFile
  #11 = Utf8               A.java
  #12 = Utf8               A
  #13 = Utf8               java/lang/Object
{
  public default java.lang.String a(java.lang.String);
    descriptor: (Ljava/lang/String;)Ljava/lang/String;
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=2, args_size=2
         0: aload_1
         1: areturn
      LineNumberTable:
        line 4: 0
    Deprecated: true
    RuntimeVisibleAnnotations:
      0: #9()
        java.lang.Deprecated
}
SourceFile: "A.java"

While the scala class bytecode looks like

Classfile /example/B.class
  Last modified 6 Jun 2023; size 687 bytes
  SHA-256 checksum 96fc9a1b3e19402a3137845a9e72d7057ebe0b7ea2913e2a0020b23949db7819
  Compiled from "B.scala"
public class example.B implements example.A
  minor version: 0
  major version: 52
  flags: (0x0021) ACC_PUBLIC, ACC_SUPER
  this_class: #2                          // example/B
  super_class: #4                         // java/lang/Object
  interfaces: 1, fields: 0, methods: 2, attributes: 4
Constant pool:
   #1 = Utf8               example/B
   #2 = Class              #1             // example/B
   #3 = Utf8               java/lang/Object
   #4 = Class              #3             // java/lang/Object
   #5 = Utf8               example/A
   #6 = Class              #5             // example/A
   #7 = Utf8               B.scala
   #8 = Utf8               Lscala/reflect/ScalaSignature;
   #9 = Utf8               bytes
  #10 = Utf8               \u0006\u0005Y1AAA\u0002\u0001\r!)1\u0003\u0001C\u0001)\t\t!IC\u0001\u0005\u0003\u001d)\u00070Y7qY\u0016\u001c\u0001aE\u0002\u0001\u000f=\u0001\"\u0001C\u0007\u000e\u0003%Q!AC\u0006\u0002\t1\fgn\u001a\u0006\u0002\u0019\u0005!!.\u0019<b\u0013\tq\u0011B\u0001\u0004PE*,7\r\u001e\t\u0003!Ei\u0011aA\u0005\u0003%\r\u0011\u0011!Q\u0001\u0007y%t\u0017\u000e\u001e \u0015\u0003U\u0001\"\u0001\u0005\u0001
  #11 = Utf8               a
  #12 = Utf8               ()Ljava/lang/String;
  #13 = Utf8               Ljava/lang/Deprecated;
  #14 = NameAndType        #11:#12        // a:()Ljava/lang/String;
  #15 = InterfaceMethodref #6.#14         // example/A.a:()Ljava/lang/String;
  #16 = Utf8               this
  #17 = Utf8               Lexample/B;
  #18 = Utf8               <init>
  #19 = Utf8               ()V
  #20 = NameAndType        #18:#19        // "<init>":()V
  #21 = Methodref          #4.#20         // java/lang/Object."<init>":()V
  #22 = Utf8               Code
  #23 = Utf8               LineNumberTable
  #24 = Utf8               LocalVariableTable
  #25 = Utf8               Deprecated
  #26 = Utf8               RuntimeVisibleAnnotations
  #27 = Utf8               SourceFile
  #28 = Utf8               ScalaInlineInfo
  #29 = Utf8               ScalaSig
{
  public java.lang.String a();
    descriptor: ()Ljava/lang/String;
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #15                 // InterfaceMethod example/A.a:()Ljava/lang/String;
         4: areturn
      LineNumberTable:
        line 3: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lexample/B;
    Deprecated: true
    RuntimeVisibleAnnotations:
      0: #13()
        java.lang.Deprecated
      1: #13()
        java.lang.Deprecated

  public example.B();
    descriptor: ()V
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #21                 // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 3: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lexample/B;
}
SourceFile: "B.scala"
RuntimeVisibleAnnotations:
  0: #8(#9=s#10)
    scala.reflect.ScalaSignature(
      bytes="\u0006\u0005Y1AAA\u0002\u0001\r!)1\u0003\u0001C\u0001)\t\t!IC\u0001\u0005\u0003\u001d)\u00070Y7qY\u0016\u001c\u0001aE\u0002\u0001\u000f=\u0001\"\u0001C\u0007\u000e\u0003%Q!AC\u0006\u0002\t1\fgn\u001a\u0006\u0002\u0019\u0005!!.\u0019<b\u0013\tq\u0011B\u0001\u0004PE*,7\r\u001e\t\u0003!Ei\u0011aA\u0005\u0003%\r\u0011\u0011!Q\u0001\u0007y%t\u0017\u000e\u001e \u0015\u0003U\u0001\"\u0001\u0005\u0001"
    )
  ScalaInlineInfo: length = 0xE (unknown attribute)
   01 00 00 02 00 12 00 13 00 00 0B 00 0C 00
  ScalaSig: length = 0x3 (unknown attribute)
   05 02 00

The same class compiled with Scala 2.13.10 gives this bytecode:

Classfile example/B.class
  Last modified 6 Jun 2023; size 683 bytes
  SHA-256 checksum a9eb082e246c34c97139612b4af5b9d7024faff53086bc427fb7159894d688b4
  Compiled from "B.scala"
public class example.B implements example.A
  minor version: 0
  major version: 52
  flags: (0x0021) ACC_PUBLIC, ACC_SUPER
  this_class: #2                          // example/B
  super_class: #4                         // java/lang/Object
  interfaces: 1, fields: 0, methods: 2, attributes: 4
Constant pool:
   #1 = Utf8               example/B
   #2 = Class              #1             // example/B
   #3 = Utf8               java/lang/Object
   #4 = Class              #3             // java/lang/Object
   #5 = Utf8               example/A
   #6 = Class              #5             // example/A
   #7 = Utf8               B.scala
   #8 = Utf8               Lscala/reflect/ScalaSignature;
   #9 = Utf8               bytes
  #10 = Utf8               \u0006\u0005Y1AAA\u0002\u0001\r!)1\u0003\u0001C\u0001)\t\t!IC\u0001\u0005\u0003\u001d)\u00070Y7qY\u0016\u001c\u0001aE\u0002\u0001\u000f=\u0001\"\u0001C\u0007\u000e\u0003%Q!AC\u0006\u0002\t1\fgn\u001a\u0006\u0002\u0019\u0005!!.\u0019<b\u0013\tq\u0011B\u0001\u0004PE*,7\r\u001e\t\u0003!Ei\u0011aA\u0005\u0003%\r\u0011\u0011!Q\u0001\u0007y%t\u0017\u000e\u001e \u0015\u0003U\u0001\"\u0001\u0005\u0001
  #11 = Utf8               a
  #12 = Utf8               ()Ljava/lang/String;
  #13 = Utf8               Ljava/lang/Deprecated;
  #14 = NameAndType        #11:#12        // a:()Ljava/lang/String;
  #15 = InterfaceMethodref #6.#14         // example/A.a:()Ljava/lang/String;
  #16 = Utf8               this
  #17 = Utf8               Lexample/B;
  #18 = Utf8               <init>
  #19 = Utf8               ()V
  #20 = NameAndType        #18:#19        // "<init>":()V
  #21 = Methodref          #4.#20         // java/lang/Object."<init>":()V
  #22 = Utf8               Code
  #23 = Utf8               LineNumberTable
  #24 = Utf8               LocalVariableTable
  #25 = Utf8               Deprecated
  #26 = Utf8               RuntimeVisibleAnnotations
  #27 = Utf8               SourceFile
  #28 = Utf8               ScalaInlineInfo
  #29 = Utf8               ScalaSig
{
  public java.lang.String a();
    descriptor: ()Ljava/lang/String;
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #15                 // InterfaceMethod example/A.a:()Ljava/lang/String;
         4: areturn
      LineNumberTable:
        line 3: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lexample/B;
    Deprecated: true
    RuntimeVisibleAnnotations:
      0: #13()
        java.lang.Deprecated

  public example.B();
    descriptor: ()V
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #21                 // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 3: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lexample/B;
}
SourceFile: "B.scala"
RuntimeVisibleAnnotations:
  0: #8(#9=s#10)
    scala.reflect.ScalaSignature(
      bytes="\u0006\u0005Y1AAA\u0002\u0001\r!)1\u0003\u0001C\u0001)\t\t!IC\u0001\u0005\u0003\u001d)\u00070Y7qY\u0016\u001c\u0001aE\u0002\u0001\u000f=\u0001\"\u0001C\u0007\u000e\u0003%Q!AC\u0006\u0002\t1\fgn\u001a\u0006\u0002\u0019\u0005!!.\u0019<b\u0013\tq\u0011B\u0001\u0004PE*,7\r\u001e\t\u0003!Ei\u0011aA\u0005\u0003%\r\u0011\u0011!Q\u0001\u0007y%t\u0017\u000e\u001e \u0015\u0003U\u0001\"\u0001\u0005\u0001"
    )
  ScalaInlineInfo: length = 0xE (unknown attribute)
   01 00 00 02 00 12 00 13 00 00 0B 00 0C 00
  ScalaSig: length = 0x3 (unknown attribute)
   05 02 00

This has only 1

    RuntimeVisibleAnnotations:
      0: #13()
        java.lang.Deprecated

SethTisue commented 1 year ago

Good catch — thanks for the report!

I wonder how widely this will be encountered in the wild. The existence of a regression puts some pressure on us to get a 2.13.12 out after too much longer, but how much pressure isn't clear to me.

SethTisue commented 1 year ago

Also not sure whether to mention it in the 2.13.11 release notes, or just assume that anyone googling the problem will land here.

I'm not sure how common it is to do this particular kind of Java reflection on compiled Scala classes. (Like, is there one or more commonly used tool or library that does it, for example.)

som-snytt commented 1 year ago

JVM spec is a bit unclear. Deprecation is "optional", and attributes with the same name and length might "clash", but it doesn't specify either behavior or restrictions. I don't see any reason to fail on duplicate Deprecation. Maybe it means "very deprecated" under -Xsource:3, for example. In any case, obviously nicer not to duplicate.

DieBauer commented 1 year ago

Also not sure whether to mention it in the 2.13.11 release notes, or just assume that anyone googling the problem will land here.

I'm not sure how common it is to do this particular kind of Java reflection on compiled Scala classes. (Like, is there one or more commonly used tool or library that does it, for example.)

I found this when using such a Scala class that I compiled with 2.13.11 (which apparently inherited a java interface with this @Deprecated method) in the Spring Boot framework.

Bean instantiation failed because they apparently hit these sun.reflect.annotation.AnnotationParser classes (for @PostConstruct and @PreDestroy annotations in this case as the stacktrace shows)

[error] Caused by: java.lang.annotation.AnnotationFormatError: Duplicate annotation for class: interface java.lang.Deprecated: @java.lang.Deprecated(forRemoval=false, since="")
[error]     at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:126)
[error]     at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:73)
[error]     at java.lang.reflect.Executable.declaredAnnotations(Executable.java:625)
[error]     at java.lang.reflect.Executable.declaredAnnotations(Executable.java:623)
[error]     at java.lang.reflect.Executable.getAnnotation(Executable.java:591)
[error]     at java.lang.reflect.Method.getAnnotation(Method.java:738)
[error]     at java.lang.reflect.AnnotatedElement.isAnnotationPresent(AnnotatedElement.java:292)
[error]     at java.lang.reflect.AccessibleObject.isAnnotationPresent(AccessibleObject.java:518)
[error]     at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.lambda$buildLifecycleMetadata$0(InitDestroyAnnotationBeanPostProcessor.java:233)
[error]     at org.springframework.util.ReflectionUtils.doWithLocalMethods(ReflectionUtils.java:324)
[error]     at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.buildLifecycleMetadata(InitDestroyAnnotationBeanPostProcessor.java:232)
[error]     at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.findLifecycleMetadata(InitDestroyAnnotationBeanPostProcessor.java:210)
[error]     at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(InitDestroyAnnotationBeanPostProcessor.java:149)
[error]     at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(CommonAnnotationBeanPostProcessor.java:305)
[error]     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyMergedBeanDefinitionPostProcessors(AbstractAutowireCapableBeanFactory.java:1116)
[error]     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)

som-snytt commented 1 year ago

I see that classfile parser sees both the DeprecatedAttr and its runtime counterpart; they are not distinguished internally, so both are emitted as DeprecatedAttrs in the class file. And I had expected something weird with default methods, oh well.

By extension, the previously ignored DeprecatedAttrs were not present as runtime-retained attributes.

som-snytt commented 1 year ago

The Scaladoc says "Annotation classes defined in Scala are not stored in classfiles in a Java-compatible manner and therefore not visible in Java reflection. In order to achieve this, the annotation has to be written in Java." I vaguely remember the interminable discussions. Now I see that the ambiguity is that Scala wants its deprecation to interoperate with the platform, so that it must be an exception to this rule. The other weirdness about default methods, and I expected one, is that an implementing class in Java does not receive a forwarding method; in Scala, it works like a trait, with a forwarder, and it is the forwarder that in turn receives the deprecated attribute. I haven't checked how other annotations are handled. But Java-defined annotations are stored according to retention policy, so Scala should at least manage that for deprecation. (There are TODOs for annots on params, etc.) Also, recall that Java has "repeatable annotations" for that use case.

seveneves commented 1 year ago

I got this problem also

scala / bug