awslabs / amazon-kinesis-producer

Amazon Kinesis Producer Library
Apache License 2.0
397 stars 330 forks source link

Dependency graph is too large and unneccessary #436

Open ryber opened 2 years ago

ryber commented 2 years ago

Hello, I recently updated from amazon-kinesis-producer 0.14.1 to 0.14.12

This dramatically increases the number of transient dependencies I am now responsible for, mostly for the Glue stuff that I don't use, but I am forced to be dependent on due to new methods on IKinesisProducer

This includes such things as:

This dependency should be option and even then, within that things like scala and kotlin should also be optional. Forcing your consumers to be dependent on something as totally unnecessary as Lombock is just kind of rude.

Full list of dependencies:

+- software.amazon.glue:schema-registry-common:jar:1.1.9:compile
[INFO] |     |  +- software.amazon.awssdk:glue:jar:2.17.201:compile
[INFO] |     |  +- software.amazon.glue:schema-registry-build-tools:jar:1.1.9:compile
[INFO] |     |  +- software.amazon.awssdk:aws-json-protocol:jar:2.17.201:compile
[INFO] |     |  +- software.amazon.awssdk:url-connection-client:jar:2.17.201:compile
[INFO] |     |  +- org.apache.avro:avro:jar:1.10.2:compile
[INFO] |     |  +- org.projectlombok:lombok:jar:1.18.20:compile
[INFO] |     |  \- org.projectlombok:lombok-utils:jar:1.18.12:compile
[INFO] |     +- software.amazon.awssdk:arns:jar:2.17.201:compile
[INFO] |     +- org.apache.kafka:kafka-clients:jar:2.8.1:compile
[INFO] |     |  +- com.github.luben:zstd-jni:jar:1.4.9-1:compile
[INFO] |     |  +- org.lz4:lz4-java:jar:1.7.1:compile
[INFO] |     |  \- org.xerial.snappy:snappy-java:jar:1.1.8.1:compile
[INFO] |     +- com.kjetland:mbknor-jackson-jsonschema_2.12:jar:1.0.39:compile
[INFO] |     |  +- org.scala-lang:scala-library:jar:2.12.10:compile
[INFO] |     |  \- org.jetbrains.kotlin:kotlin-scripting-compiler-embeddable:jar:1.3.50:compile
[INFO] |     |     \- org.jetbrains.kotlin:kotlin-scripting-compiler-impl-embeddable:jar:1.3.50:runtime
[INFO] |     |        +- org.jetbrains.kotlin:kotlin-scripting-common:jar:1.3.50:runtime
[INFO] |     |        +- org.jetbrains.kotlin:kotlin-scripting-jvm:jar:1.3.50:runtime
[INFO] |     |        |  \- org.jetbrains.kotlin:kotlin-script-runtime:jar:1.3.50:runtime
[INFO] |     |        \- org.jetbrains.kotlinx:kotlinx-coroutines-core:jar:1.1.1:runtime
[INFO] |     +- io.github.classgraph:classgraph:jar:4.8.120:compile
[INFO] |     +- com.github.erosb:everit-json-schema:jar:1.12.2:compile
[INFO] |     |  +- commons-validator:commons-validator:jar:1.6:compile
[INFO] |     |  |  \- commons-digester:commons-digester:jar:1.8.1:compile
[INFO] |     |  +- com.damnhandy:handy-uri-templates:jar:2.1.8:compile
[INFO] |     |  \- com.google.re2j:re2j:jar:1.3:compile
[INFO] |     +- com.squareup.wire:wire-schema:jar:3.7.1:compile
[INFO] |     |  +- org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.10:compile
[INFO] |     |  +- com.squareup.wire:wire-runtime:jar:3.7.1:compile
[INFO] |     |  \- org.jetbrains.kotlin:kotlin-stdlib-common:jar:1.4.10:compile
[INFO] |     +- com.squareup.wire:wire-compiler:jar:3.7.1:compile
[INFO] |     |  +- com.squareup.wire:wire-kotlin-generator:jar:3.7.1:compile
[INFO] |     |  |  +- com.squareup:kotlinpoet:jar:1.7.2:compile
[INFO] |     |  |  |  \- org.jetbrains.kotlin:kotlin-reflect:jar:1.4.10:runtime
[INFO] |     |  |  \- com.squareup.wire:wire-grpc-server-generator:jar:3.7.1:runtime
[INFO] |     |  +- org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.4.10:compile
[INFO] |     |  |  \- org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:1.4.10:compile
[INFO] |     |  +- com.squareup.wire:wire-java-generator:jar:3.7.1:runtime
[INFO] |     |  |  \- com.squareup:javapoet:jar:1.13.0:runtime
[INFO] |     |  +- com.squareup.wire:wire-swift-generator:jar:3.7.1:runtime
[INFO] |     |  |  \- io.outfoxx:swiftpoet:jar:1.0.0:runtime
[INFO] |     |  +- com.squareup.wire:wire-profiles:jar:3.7.1:runtime
[INFO] |     |  +- org.jetbrains.kotlinx:kotlinx-serialization-core-jvm:jar:1.0.1:runtime
[INFO] |     |  \- com.charleskorn.kaml:kaml:jar:0.20.0:runtime
[INFO] |     |     \- org.snakeyaml:snakeyaml-engine:jar:2.1:runtime
[INFO] |     +- com.google.api.grpc:proto-google-common-protos:jar:2.7.4:compile
[INFO] |     \- com.google.jimfs:jimfs:jar:1.1:compile
ryber commented 2 years ago

For right now I have made by own stub classes of the following to allow myself to exclude the glue lib and continue to use this library for my original purpose:

com.amazonaws.services.schemaregistry.common.Schema com.amazonaws.services.schemaregistry.common.configs.GlueSchemaRegistryConfiguration com.amazonaws.services.schemaregistry.serializers.GlueSchemaRegistrySerializer com.amazonaws.services.schemaregistry.serializers.GlueSchemaRegistrySerializerImpl

Some of these I had to stub out the constructors and a method or two just so I could new up KinesisProducer.

VladimirPchelko commented 2 years ago

Using Lombock is sabotage - it greatly interferes with navigation through the code as well as debugging ... "invisible" getters and setters, as well as constructors! For now many AWS libraries uses this Lombock ... this is very bad...

KPL is not a product that you can take and just use. I spent many hours debugging and researching the code!

Especially this https://github.com/awslabs/amazon-kinesis-producer/issues/48 (our team freaked out from that issue)