camel-minio-sink-connector giving library issue

[shubhamcoc]

Hi, I am trying to run camel-minio-sink-connector plugin with the following config:

name=s3-sink-connector connector.class=org.apache.camel.kafkaconnector.miniosink.CamelMiniosinkSinkConnector tasks.max=1 key.converter=org.apache.kafka.connect.storage.StringConverter value.converter=org.apache.kafka.connect.storage.StringConverter topics=kafkatopic-sample camel.kamelet.minio-sink.bucketName=test camel.kamelet.minio-sink.accessKey=XXXXXXXXXXXXXXXXXXXXXX camel.kamelet.minio-sink.secretKey=XXXXXXXXXXXXXXXXXXXXXX camel.kamelet.minio-sink.endpoint=https://minio-url errors.log.enable=true errors.log.include.messages=true

but I am getting issue in the logs as attached below

Do anyone know, how to solve this OkHttp library issue? Any help is appreciated. Thanks in advance.

[oscerd]

What is the Ckc version of the connector? What is inside your plugin path? You can try to add the okhttp jar with 4.8.1 in your plugin path in the connector folder.

[shubhamcoc]

Thanks @oscerd for replying,

CKC is 3.20.6 and inside my plugin path, I have 6 plugins,

1. camel-aws-s3-sink-kafka-connector
2. camel-aws-s3-source-kafka-connector
3. camel-minio-source-kafka-connector
4. camel-minio-sink-kafka-connector
5. camel-kafka-ssl-source-kafka-connector
6. camel-kafka-ssl-sink-kafka-connector

All plugins are having 3.20.6 version. I believe aws-s3 plugin is mainly for aws s3 bucket more than minio s3 bucket not hosted on aws.

kafka-ssl-source and kafka-ssl-sink, I am not adding to connect to kafka cluster, to read the data from kafka topic.

I will try to add okhttp jar 4.8.1 into the connector but will the connector pick it up, need to check.

[oscerd]

Can you list the jars in the minio sink connector folder? This kind of problem should come up even in other camel runtimes I guess

[shubhamcoc]

No in other plugin that I am running is kafka-ssl-sink, and it is able to run properly without any trouble, and error is when minio plugin is try to connect to minio instance. Here is the content of the folder and I can see okhttp-4.10.0.jar is there in the directory

├── annotations-13.0.jar ├── apicurio-registry-common-1.3.2.Final.jar ├── apicurio-registry-rest-client-1.3.2.Final.jar ├── apicurio-registry-utils-converter-1.3.2.Final.jar ├── apicurio-registry-utils-serde-1.3.2.Final.jar ├── avro-1.11.1.jar ├── bcprov-jdk15on-1.69.jar ├── camel-api-3.20.6.jar ├── camel-base-3.20.6.jar ├── camel-base-engine-3.20.6.jar ├── camel-bean-3.20.6.jar ├── camel-browse-3.20.6.jar ├── camel-cluster-3.20.6.jar ├── camel-controlbus-3.20.6.jar ├── camel-core-3.20.6.jar ├── camel-core-engine-3.20.6.jar ├── camel-core-languages-3.20.6.jar ├── camel-core-model-3.20.6.jar ├── camel-core-processor-3.20.6.jar ├── camel-core-reifier-3.20.6.jar ├── camel-dataformat-3.20.6.jar ├── camel-dataset-3.20.6.jar ├── camel-direct-3.20.6.jar ├── camel-directvm-3.20.6.jar ├── camel-dsl-support-3.20.6.jar ├── camel-file-3.20.6.jar ├── camel-health-3.20.6.jar ├── camel-kafka-3.20.6.jar ├── camel-kafka-connector-3.20.6.jar ├── camel-kamelet-3.20.6.jar ├── camel-language-3.20.6.jar ├── camel-log-3.20.6.jar ├── camel-main-3.20.6.jar ├── camel-management-api-3.20.6.jar ├── camel-minio-3.20.6.jar ├── camel-minio-sink-kafka-connector-3.20.6.jar ├── camel-mock-3.20.6.jar ├── camel-ref-3.20.6.jar ├── camel-rest-3.20.6.jar ├── camel-saga-3.20.6.jar ├── camel-scheduler-3.20.6.jar ├── camel-seda-3.20.6.jar ├── camel-stub-3.20.6.jar ├── camel-support-3.20.6.jar ├── camel-timer-3.20.6.jar ├── camel-tooling-model-3.20.6.jar ├── camel-util-3.20.6.jar ├── camel-util-json-3.20.6.jar ├── camel-validator-3.20.6.jar ├── camel-vm-3.20.6.jar ├── camel-xml-io-util-3.20.6.jar ├── camel-xml-jaxb-3.20.6.jar ├── camel-xml-jaxp-3.20.6.jar ├── camel-xpath-3.20.6.jar ├── camel-xslt-3.20.6.jar ├── camel-yaml-dsl-3.20.6.jar ├── camel-yaml-dsl-common-3.20.6.jar ├── camel-yaml-dsl-deserializers-3.20.6.jar ├── commons-compress-1.21.jar ├── connect-json-2.6.0.jar ├── converter-jackson-2.9.0.jar ├── guava-20.0.jar ├── jackson-annotations-2.14.3.jar ├── jackson-core-2.14.3.jar ├── jackson-databind-2.14.3.jar ├── jackson-dataformat-avro-2.14.3.jar ├── jackson-datatype-jdk8-2.10.2.jar ├── jakarta.activation-1.2.2.jar ├── jakarta.activation-api-1.2.2.jar ├── jakarta.xml.bind-api-2.3.3.jar ├── javax.annotation-api-1.3.2.jar ├── jaxb-core-2.3.0.jar ├── jaxb-impl-2.3.3.jar ├── jboss-jaxrs-api_2.1_spec-2.0.1.Final.jar ├── jctools-core-3.3.0.jar ├── kotlin-reflect-1.3.20.jar ├── kotlin-stdlib-1.6.20.jar ├── kotlin-stdlib-common-1.5.31.jar ├── kotlin-stdlib-jdk7-1.5.31.jar ├── kotlin-stdlib-jdk8-1.5.31.jar ├── LICENSE.txt ├── medeia-validator-core-1.1.1.jar ├── medeia-validator-jackson-1.1.1.jar ├── minio-8.4.6.jar ├── NOTICE.txt ├── okhttp-4.10.0.jar ├── okio-jvm-3.0.0.jar ├── protobuf-java-3.13.0.jar ├── README.adoc ├── retrofit-2.9.0.jar ├── simple-xml-safe-2.7.1.jar └── snakeyaml-engine-2.3.jar

[oscerd]

I'll try to have a look

[shubhamcoc]

Thanks @oscerd , FYI I am running standalone instance of apache kafka connect. I hope that is not an issue right? and one more query, asw-s3-sink is meant for s3 buckets in aws instance only. Is this understanding correct?

[oscerd]

It's fine. What is the kafka version?

Yes, S3 is for S3 service in AWS

[shubhamcoc]

kafka version is 3.1.2

[shubhamcoc]

Hi @oscerd, any update on this?

[oscerd]

I'll have a look when I have time. I don't have an ETA.

[shubhamcoc]

ok sure, let me know, once you check it. Thanks

[shubhamcoc]

Hi @oscerd, I am able to figure it out, the problem was, other plugins were having okhttp 3.14.9 version, as a result, it was picking the older version, and another thing is we don't need Kafka plugins separately for connecting with Kafka, in order to connect kafka with s3, only Minio sink and source plugins were enough. Thanks for the help. I need one more clarification, can you point out where is the source code of the plugins?

[oscerd]

The plugins are generate starting from kamelets: https://github.com/apache/camel-kamelets/tree/main/kamelets

Usually is better to have one single folder per connector in your plugin path, instead of having multiple in the same folder.

[shubhamcoc]

Noted. Thanks for the guidance, currently I have kept it in separate folders.

So we can't see the code for plugins as it is generated? Thanks for the info.

[oscerd]

The generated connectors are here: https://github.com/apache/camel-kafka-connector/tree/main/connectors

[shubhamcoc]

ok thanks, will check it, closing the ticket for now. As the problem is solved.