search

confluentinc / cp-docker-images

[DEPRECATED] Docker images for Confluent Platform.
Apache License 2.0
1.14k stars 704 forks source link

Unable to build image with local connector #776

Open wanglisheng0111 opened 5 years ago

wanglisheng0111 commented 5 years ago

Hi,

I'm a newer to Docker, i face a issue when i try to create a image with local connector. i follow this instruction i'm using Mac OS, content of Dockerfile as below:

`FROM confluentinc/cp-kafka-connect-base:5.3.0

COPY folder/kafka-connect-solr-sink-0.0.1.zip /tmp/kafka-connect-solr-sink-0.0.1.zip

RUN confluent-hub install --no-prompt /tmp/kafka-connect-solr-sink-0.0.1.zip when i rundocker build . -t my-custom-image:1.0.0`, i got below error:

Sending build context to Docker daemon 6.876MB Step 1/3 : FROM confluentinc/cp-kafka-connect-base:5.3.0 ---> 7a97ee221e04 Step 2/3 : COPY folder/kafka-connect-solr-sink-0.0.1.zip /tmp/kafka-connect-solr-sink-0.0.1.zip ---> Using cache ---> c2300114a994 Step 3/3 : RUN confluent-hub install --no-prompt /tmp/kafka-connect-solr-sink-0.0.1.zip ---> Running in ef10d1feadac Running in a "--no-prompt" mode Failed to unzip '/tmp/kafka-connect-solr-sink-0.0.1.zip' into '/tmp/confluent-hub-tmp2720761103257567924' String index out of range: -1

Error: Unknown error The command '/bin/sh -c confluent-hub install --no-prompt /tmp/kafka-connect-solr-sink-0.0.1.zip' returned a non-zero code: 7

BTW, the version of the connect-api that my local connector builded with dependency is 1.1.0.

anyone can kindly help on that?

Thanks in advance!

sajhak commented 5 years ago

Can't see any issue with your Dockerfile, i was able to get an image built using the same content with bkatwal-bkatwal-kafka-connect-solr-sink-2.0

From where did you download kafka-connect-solr-sink-0.0.1.zip ?

Only issue i can think is whether your zip file is corrupted, or can you unzip in your local machine? (outside of docker)

If possible please attach the zip file or paste a link of the zip file

wanglisheng0111 commented 5 years ago

Hi Sajhak Thanks for your response. kafka-connect-solr-sink-0.0.1.zip is built and compressed by myself. the kafka-connect-solr-sink-0.0.1.jar is runnable as i can run on confluent which deployed on VM. i can unzip on my laptop. attached is zip. kafka-connect-solr-sink-0.0.1.zip Thanks. Lisheng

sajhak commented 5 years ago

Hi Lisheng,

The command "confluent-hub install" works only for the connectors which are packaged as per the confluent hub component archive format. https://docs.confluent.io/current/connect/managing/confluent-hub/component-archive.html#confluent-hub-component-archive

I can see your zip file (kafka-connect-solr-sink-0.0.1.zip) is not in that format, rather that zip file contains an Uber jar (with all the dependencies etc..)

In order to install connectors in that format, you simply have to copy to the plugin path. (Copy the zip file or the jar file to any location you want, and when starting up the container, set the environment variable to point to that path)

Dockerfile FROM confluentinc/cp-kafka-connect-base:5.3.0 COPY folder/kafka-connect-solr-sink-0.0.1.zip /usr/share/java

When starting up the container, provide the env variable for eg: with docker-compose ... environment: CONNECT_PLUGIN_PATH: "/usr/share/java"

https://docs.confluent.io/current/connect/userguide.html#installing-plugins

Hope this helps

wanglisheng0111 commented 5 years ago

Hi Sajhak

Thank you so much for helping, your explanation is very clear and helpful.

i will try with your instruction.

Thanks a lot! Lisheng