aws-samples / amazon-kinesis-data-analytics-examples

Example applications in Java, Python and SQL for Kinesis Data Analytics, demonstrating sources, sinks, and operators.
MIT No Attribution
139 stars 87 forks source link

Scala NoSuchMethodError: 'scala.collection.mutable.Buffer (and all other Scala packages) #49

Closed GensaGames closed 1 year ago

GensaGames commented 1 year ago

Steps to reproduce:

    import scala.jdk.CollectionConverters._

    val a = util.Arrays.asList(1,2,3,4,5)
    val b = a.asScala.toList.map { x =>
      x^2
    }

    log.info(s"Does it work?: $b")

Error in runtime: java.lang.NoSuchMethodError: 'scala.collection.mutable.Buffer scala.collection.convert.AsScalaConverters.asScala$(scala.collection.convert.AsScalaConverters, java.util.List)'

Please Note! While checking jar package you can find, that all the required libraries already included, for ex. scala.collection.mutable.Buffer and scala.collection.convert.AsScalaConverters already part of the jar and runtime. You can check jar package with command jar tf <file>. However it doesn't work when you build and run in Kinesis Flink application.

GensaGames commented 1 year ago

The issue was in Flink version on AWS. Starting from 1.15 scala interpreter was included, but prior versions doesn't work with scala versions.