search

apache / incubator-streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://streampark.apache.org/
Apache License 2.0
3.91k stars 1.01k forks source link

[Bug] maven conflict #2746

Open learner1212 opened 1 year ago

learner1212 commented 1 year ago

Search before asking

Java Version

1.8

Scala Version

2.12.x

StreamPark Version

2.1.0

Flink Version

1.16.1

deploy mode

yarn-application

What happened

when paste maven dependencies, throw NoSuchMethodError.

in streampark-flink-packer module, maven-aether-provider.3.3.9.jar include aether-api.1.0.2.v20150114.jar but we want 1.1.0 version

Error Exception

02:31:37.362 [streampark-build-pipeline-executor-1] ERROR org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline - [StreamPark] [streampark-packer] building pipeline has failed. | appName=load_uv
java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.apache.maven.model.ReportSet.isInherited()Z
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:206)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$launch$1(BuildPipeline.scala:136)
    at scala.util.Try$.apply(Try.scala:213)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch(BuildPipeline.scala:129)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch$(BuildPipeline.scala:127)
    at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.launch(FlinkYarnApplicationBuildPipeline.scala:32)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.NoSuchMethodError: org.apache.maven.model.ReportSet.isInherited()Z
    at org.apache.maven.model.merge.MavenModelMerger.mergeReportPlugin_ReportSets(MavenModelMerger.java:603)
    at org.apache.maven.model.merge.ModelMerger.mergeReportPlugin(ModelMerger.java:1328)
    at org.apache.maven.model.inheritance.DefaultInheritanceAssembler$InheritanceModelMerger.mergeReporting_Plugins(DefaultInheritanceAssembler.java:295)
    at org.apache.maven.model.merge.ModelMerger.mergeReporting(ModelMerger.java:1260)
    at org.apache.maven.model.merge.ModelMerger.mergeModelBase_Reporting(ModelMerger.java:653)
    at org.apache.maven.model.merge.ModelMerger.mergeModelBase(ModelMerger.java:522)
    at org.apache.maven.model.merge.ModelMerger.mergeModel(ModelMerger.java:116)
    at org.apache.maven.model.merge.MavenModelMerger.mergeModel(MavenModelMerger.java:75)
    at org.apache.maven.model.merge.ModelMerger.merge(ModelMerger.java:111)
    at org.apache.maven.model.inheritance.DefaultInheritanceAssembler.assembleModelInheritance(DefaultInheritanceAssembler.java:58)
    at org.apache.maven.model.building.DefaultModelBuilder.assembleInheritance(DefaultModelBuilder.java:721)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:377)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:287)
    at org.apache.streampark.flink.packer.maven.MavenTool$.$anonfun$resolveArtifacts$4(MavenTool.scala:189)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
    at scala.collection.immutable.Set$Set4.foreach(Set.scala:208)
    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
    at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:51)
    at scala.collection.SetLike.map(SetLike.scala:104)
    at scala.collection.SetLike.map$(SetLike.scala:104)
    at scala.collection.AbstractSet.map(Set.scala:51)
    at org.apache.streampark.flink.packer.maven.MavenTool$.resolveArtifacts(MavenTool.scala:189)
    at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.$anonfun$buildProcess$4(FlinkYarnApplicationBuildPipeline.scala:60)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$execStep$1(BuildPipeline.scala:102)
    at scala.util.Try$.apply(Try.scala:213)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep(BuildPipeline.scala:97)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep$(BuildPipeline.scala:96)
    at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.execStep(FlinkYarnApplicationBuildPipeline.scala:32)
    at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.buildProcess(FlinkYarnApplicationBuildPipeline.scala:58)
    at org.apache.streampark.flink.packer.pipeline.impl.FlinkYarnApplicationBuildPipeline.buildProcess(FlinkYarnApplicationBuildPipeline.scala:32)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:134)
    at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:133)
    ... 4 common frames omitted

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

shenyi0828 commented 7 months ago

I don't think streampark can resovle maven dependency in depth.
When I have `

com.ververica flink-connector-mysql-cdc 2.3.0 ` MavenTool.scala will only resolve 1st depth deps, which are `flink-connector-debezium`, `debezium-connector-mysql` . Stop digging further deps.