DataReply / kafka-connect-mongodb

Apache License 2.0
129 stars 61 forks source link

Failed to find any class that implements Connector #23

Open vlad0337187 opened 6 years ago

vlad0337187 commented 6 years ago

Hello.

I placed .jar (with or without dependencies) into plugins folder, but when I'm trying to create connector (via post request) - it gives me an error:

{
    "error_code": 500,
    "message": "Failed to find any class that implements Connector and which name matches org.apache.kafka.connect.mongodb.MongodbSourceConnector, available connectors are: PluginDesc{klass=class com.datamountaineer.streamreactor.connect.mongodb.sink.MongoSinkConnector, name='com.datamountaineer.streamreactor.connect.mongodb.sink.MongoSinkConnector', version='null', encodedVersion=null, type=connector, typeName='connector', location='file:/usr/share/java/connector-plugins/kafka-connect-mongodb-0.3.0-3.3.0-all.jar'}, PluginDesc{klass=class io.confluent.connect.elasticsearch.ElasticsearchSinkConnector, name='io.confluent.connect.elasticsearch.ElasticsearchSinkConnector', version='4.0.0', encodedVersion=4.0.0, type=sink, typeName='sink', location='file:/usr/share/java/connector-plugins/connector-elasticsearch/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mongodb.MongodbSinkConnector, name='org.apache.kafka.connect.mongodb.MongodbSinkConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=sink, typeName='sink', location='file:/usr/share/java/connector-plugins/connect-mongodb-1.1-jar-with-dependencies.jar'}, PluginDesc{klass=class org.apache.kafka.connect.mongodb.MongodbSourceConnector, name='org.apache.kafka.connect.mongodb.MongodbSourceConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='file:/usr/share/java/connector-plugins/connect-mongodb-1.1-jar-with-dependencies.jar'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='1.0.0-cp1', encodedVersion=1.0.0-cp1, type=source, typeName='source', location='classpath'}"
}

The classname, which I specified in "connector.class" property is present in list below (list of available connectors).

For comparison, Mongo sync from this pack: https://github.com/Landoop/stream-reactor works (and creates) well. kafka-connect-mongo doesn't creates with same error too.

Docker log: https://gist.github.com/vlad1777d/7cd9a8744fe2f6ca5e8c0a3f57834d89 Docker Compose .yml file: docker-compose.yml.txt

Build was successful:

[INFO] Building jar: /Users/vlad/Downloads/kafka-connect-mongodb-master/target/connect-mongodb-1.1-jar-with-dependencies.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:12 min
[INFO] Finished at: 2017-12-27T18:45:26+02:00
[INFO] Final Memory: 28M/208M
[INFO] ------------------------------------------------------------------------
beijing1 commented 6 years ago

Anybody is working on this issue?

ruileal commented 6 years ago

I am having the exact same issue. The same connector said to not be found is in the list of available connectors.

ruileal commented 6 years ago

Found the problem. As the confluent documentation writes: "To install a new plugin place it in the CLASSPATH of the Kafka Connect process" export CLASSPATH=/path/to/my/connectors/*

umeshmishra099 commented 6 years ago

I am facing same issue. i am using windows SET CLASSPATH = "location of these jars connect-mongodb-1.1-jar-with-dependencies.jar , connect-mongodb-1.1.jar".

and running below command: bin\windows\connect-standalone worker.properties mongo-connector.properties

Still failing with below error:

java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.kafka.connect s are: org.apache.kafka.connect.file.FileStreamSourceConnector, org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.tools fiableSourceConnector, org.apache.kafka.connect.source.SourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:80) at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:67) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:97) Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.kafka.connect.mongodb.MongodbSourceConnecto t.file.FileStreamSourceConnector, org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.tools.MockConnector, org.apache.kaf che.kafka.connect.source.SourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache.kafka.connect.tools.Verifiabl at org.apache.kafka.connect.runtime.Worker.getConnectorClass(Worker.java:226) at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:166) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:250) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:164) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:94) [2018-05-29 23:53:57,396] ERROR Error shutting down connector mongodb-source-connector: (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:87) org.apache.kafka.connect.errors.ConnectException: Connector mongodb-source-connector not found in this worker. at org.apache.kafka.connect.runtime.Worker.stopConnector(Worker.java:282) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.stop(StandaloneHerder.java:85) at org.apache.kafka.connect.runtime.Connect.stop(Connect.java:71) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:101)

eugenemiretsky commented 6 years ago

Getting the same issue. Did anybody find a solution?

sandeepbishnoi commented 6 years ago

I got the similar issue while installing a custom connector. To resolve that I created another directory customconnector inside plugin.path /opt/connectors. Extracted contents of shaded jar into the customconnector directory. plugin.path=/opt/connectors

msemn commented 5 years ago

"plugin.path=/path/to/your/connectors" should also be set in "/path/to/kafka/config/connect-distributed.properties"

hamzanasirr commented 4 years ago

I am having the same error. Anybody solved the issue? I am having this error on Debezium's MongoDB Source Connector.

hamzanasirr commented 4 years ago

I am facing same issue. i am using windows SET CLASSPATH = "location of these jars connect-mongodb-1.1-jar-with-dependencies.jar , connect-mongodb-1.1.jar".

and running below command: bin\windows\connect-standalone worker.properties mongo-connector.properties

Still failing with below error:

java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.kafka.connect s are: org.apache.kafka.connect.file.FileStreamSourceConnector, org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.tools fiableSourceConnector, org.apache.kafka.connect.source.SourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:80) at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:67) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:97) Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.kafka.connect.mongodb.MongodbSourceConnecto t.file.FileStreamSourceConnector, org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.tools.MockConnector, org.apache.kaf che.kafka.connect.source.SourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache.kafka.connect.tools.Verifiabl at org.apache.kafka.connect.runtime.Worker.getConnectorClass(Worker.java:226) at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:166) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:250) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:164) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:94) [2018-05-29 23:53:57,396] ERROR Error shutting down connector mongodb-source-connector: (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:87) org.apache.kafka.connect.errors.ConnectException: Connector mongodb-source-connector not found in this worker. at org.apache.kafka.connect.runtime.Worker.stopConnector(Worker.java:282) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.stop(StandaloneHerder.java:85) at org.apache.kafka.connect.runtime.Connect.stop(Connect.java:71) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:101)

Hi, I know I am asking quite late (2 years late) but have you been able to solve this problem?

Thanks!

girionis commented 4 years ago

Hope I am not too late into this. If you're using windows make sure your plugin.path uses *nix-based path, now Windows based. Do not use plugin.path=C:\Apps\kafka_2.12-2.5.0\plugins but use plugin.path=C:/Apps/kafka_2.12-2.5.0/plugins Took me a while to figure this out.

capndave commented 3 years ago

I'm having the same issue as @hamzanasirr, trying to run mongodb-kafka-connect-mongodb-1.3.0-all.jar on Windows. Any help would be appreciated!

thinkinerp commented 3 years ago

the last choice is to copy the jars to $KAFAK_HOME/libs ,and that helped me.

sanraj25 commented 3 years ago

Adding the trailing slash at the end of the plugin path variable solves the issue. plugin.path=C:/Application/kafka/plugins/

kenneth-pro commented 3 years ago

Any working solutions?

Ting-Kim commented 2 years ago

In my case, I wrote ~/somewhere/somewhere_detail into plugin.path on connect-distributed.properties file. After I change that to absolute path(/usr/local/somewhere/somwhere_detail), it was resolved. (my case was macOS)

if you wrote ~ in this file, why don't you try to replace ~ to detail path?

JaydenHu6 commented 2 years ago

I got same issue. I set plugin.path=/root/develop/opt/kafka_2.12-3.3.1/plugins/debezium-connector-mysql in connect-distributed.properties and restart my kafka connect using bin/connect-distributed.sh config/connect-distributed.properties but still does't works.

JaydenHu6 commented 2 years ago

I got same issue. I set plugin.path=/root/develop/opt/kafka_2.12-3.3.1/plugins/debezium-connector-mysql in connect-distributed.properties and restart my kafka connect using bin/connect-distributed.sh config/connect-distributed.properties but still does't works.

I solved it. my kafka version is 3.3.1,when debezium version is 2.0 it doesn't works. I replace my debezium version to 1.9 it works well.