search

FasterXML / jackson-module-scala

Add-on module for Jackson (https://github.com/FasterXML/jackson) to support Scala-specific datatypes
Apache License 2.0
500 stars 141 forks source link

Dependency on .class and .tasty files causes footprint bloat for GraalVM NativeImage #642

Open AlanBurlison opened 1 year ago

AlanBurlison commented 1 year ago

Both .class and .tasty files are read at run time, e.g.

.class file access stack trace

    at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1733)
    at com.thoughtworks.paranamer.BytecodeReadingParanamer.getClassAsStream(BytecodeReadingParanamer.java:131)
    at com.thoughtworks.paranamer.BytecodeReadingParanamer.getClassAsStream(BytecodeReadingParanamer.java:124)
    at com.thoughtworks.paranamer.BytecodeReadingParanamer.lookupParameterNames(BytecodeReadingParanamer.java:92)
    at com.thoughtworks.paranamer.CachingParanamer.lookupParameterNames(CachingParanamer.java:90)
    at com.fasterxml.jackson.module.scala.introspect.JavaParameterIntrospector$.getCtorParamNames$$anonfun$1(JavaParameterIntrospector.scala:13)
    at com.fasterxml.jackson.module.scala.introspect.JavaParameterIntrospector$$$Lambda$310/0x0000000800edafc0.apply(Unknown Source:-1)
    at scala.util.Try$.apply(Try.scala:210)
    at com.fasterxml.jackson.module.scala.introspect.JavaParameterIntrospector$.getCtorParamNames(JavaParameterIntrospector.scala:13)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.getCtorParams(BeanIntrospector.scala:246)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$2(BeanIntrospector.scala:53)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$$$Lambda$309/0x0000000800ed9d08.apply(Unknown Source:-1)
    at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:118)
    at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:105)
    at scala.collection.immutable.Vector.flatMap(Vector.scala:113)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.findConstructorParam$1(BeanIntrospector.scala:53)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$5$$anonfun$5(BeanIntrospector.scala:195)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$$$Lambda$307/0x0000000800ed36a0.apply(Unknown Source:-1)
    at scala.collection.ArrayOps$.map$extension(ArrayOps.scala:932)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$5(BeanIntrospector.scala:195)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$$$Lambda$288/0x0000000800ecf4b0.apply(Unknown Source:-1)
    at scala.collection.immutable.List.flatMap(List.scala:293)
    at scala.collection.immutable.List.flatMap(List.scala:79)
    at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.apply(BeanIntrospector.scala:195)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$._descriptorFor(ScalaAnnotationIntrospectorModule.scala:196)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$.fieldName(ScalaAnnotationIntrospectorModule.scala:207)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$.findImplicitPropertyName(ScalaAnnotationIntrospectorModule.scala:41)
    at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findImplicitPropertyName(AnnotationIntrospectorPair.java:488)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addFields(POJOPropertiesCollector.java:556)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:445)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getPropertyMap(POJOPropertiesCollector.java:405)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getProperties(POJOPropertiesCollector.java:247)
    at com.fasterxml.jackson.databind.introspect.BasicBeanDescription._properties(BasicBeanDescription.java:164)
    at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findProperties(BasicBeanDescription.java:239)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCreatorsFromProperties(BasicDeserializerFactory.java:317)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:271)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:222)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:262)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:151)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:415)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:350)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:654)
    at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4956)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4826)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3785)

.tasty file access stack trace

    at java.lang.Class.getResource(Class.java:2939)
    at com.fasterxml.jackson.module.scala.util.TastyUtil$.hasTastyFile(TastyUtil.scala:22)
    at com.fasterxml.jackson.module.scala.util.ClassW.extendsScalaClass(Classes.scala:16)
    at com.fasterxml.jackson.module.scala.util.ClassW.extendsScalaClass$(Classes.scala:8)
    at com.fasterxml.jackson.module.scala.util.ClassW$$anon$1.extendsScalaClass(Classes.scala:39)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$._descriptorFor(ScalaAnnotationIntrospectorModule.scala:186)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$.fieldName(ScalaAnnotationIntrospectorModule.scala:207)
    at com.fasterxml.jackson.module.scala.introspect.ScalaAnnotationIntrospector$.findImplicitPropertyName(ScalaAnnotationIntrospectorModule.scala:41)
    at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findImplicitPropertyName(AnnotationIntrospectorPair.java:488)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addFields(POJOPropertiesCollector.java:556)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:445)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getPropertyMap(POJOPropertiesCollector.java:405)
    at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getProperties(POJOPropertiesCollector.java:247)
    at com.fasterxml.jackson.databind.introspect.BasicBeanDescription._properties(BasicBeanDescription.java:164)
    at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findProperties(BasicBeanDescription.java:239)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCreatorsFromProperties(BasicDeserializerFactory.java:317)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:271)
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:222)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:262)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:151)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:415)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:350)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:621)
    at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.createContextual(ObjectArrayDeserializer.java:147)
    at com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:867)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:659)
    at com.fasterxml.jackson.databind.ObjectReader._prefetchRootDeserializer(ObjectReader.java:2430)
    at com.fasterxml.jackson.databind.ObjectReader.<init>(ObjectReader.java:194)
    at com.fasterxml.jackson.databind.ObjectMapper._newReader(ObjectMapper.java:780)
    at com.fasterxml.jackson.databind.ObjectMapper.readerForArrayOf(ObjectMapper.java:4303)

This isn't an issue for normal JVM usage, but if the application is built with GraalVM Native Image it results in both the .class and .tasty files being included in the resulting executable, leading to increased footprint.

With the correct NativeImage reflection configuration the image does work so I'd hesitate to call this an outright bug, but the requirement to access both .class and .tasty files at runtime is an issue if Native Image size is a concern.

There may be nothing that can be done about the need to include .tasty files for runtime access. I may be wrong, but from a cursory look it appears that com.thoughtworks.paranamer is being used to replicate functionality such as reading parameter names that can be done directly?

pjfanning commented 1 year ago

I don't much about building GraalVM native images. Can you find a way to exclude the .tasty files when you build your on images?

pjfanning commented 1 year ago

The tasty lookup can be disabled at runtime by calling this API

supportScala3Classes(support: Boolean): Unit

AlanBurlison commented 1 year ago

Can you find a way to exclude the .tasty files when you build your on images?

Yes, that's easy enough to do, but then you get run-time errors because they aren't there.

Native Image depends on knowing about reflective accesses at build time, the usual way of determining that is to run your app in normal JVM mode with a tracing agent that intercepts all the reflective calls and writes them to a config file for use when building the image.

The tasty lookup can be disabled at runtime by calling this API supportScala3Classes(...

Presumably that will break Scala3 support, which I'm using?

I've done some more digging, I'm also using com.github.pjfanning.scala3-reflection and the associated compiler plugin, plus com.fasterxml.jackson.module.scala.ClassTagExtensions. If I remove all those, from first look it appears that the access to the .class & .tasty files goes away and for my app at least, things still work - but as I'm not entirely clear how all the bits mesh together in the first place...

pjfanning commented 1 year ago

Frankly if you want tiny images then you should probably consider not using Scala. Scala is not tailored for the tiny image use case.

AlanBurlison commented 1 year ago

I realise that, but when you have hard size limits on the resulting image, pulling in .class and .tasty files is significant.

As I said I'd hesitate to call this a bug but it's taken me a while to figure out why those files were being included in the Native Image, in the pure-Java case .class files aren't pulled in. If nothing else this info will help others trying to do the same thing.