search

Aiven-Open / opensearch-connector-for-apache-kafka

Aiven's OpenSearch® Connector for Apache Kafka®
Apache License 2.0
64 stars 35 forks source link

Getting error of NoClassDefFoundError Could not initialize class io.aiven.kafka.connect.opensearch.OpensearchSinkConnectorConfig #186

Open Hardik-Parikh opened 1 year ago

Hardik-Parikh commented 1 year ago 
// POST Request http://localhost:8083/connectors

// BODY (RAW JSON)
{
    "name": "first-opensearch-connector",
    "config": {
        "connector.class": "io.aiven.kafka.connect.opensearch.OpensearchSinkConnector",
        "connection.url": "http://localhost:9200/",
        "tasks.max": "1",
        "topics": "my_topic",
        "key.ignore": "true",
        "name": "first-opensearch-connector",
        "connection.username": "admin",
        "connection.password": "admin",
        "value.converter": "org.apache.kafka.connect.json.JsonConverter",
        "value.converter.schemas.enable": "false",
        "schema.ignore": "true",
        "type.name": "_doc"
    }
}
Error 500 Request failed.

HTTP ERROR 500 Request failed.

URI: /connectors/
STATUS: 500
MESSAGE: Request failed.
SERVLET: org.glassfish.jersey.servlet.ServletContainer-22c75c01
Powered by Jetty:// 9.4.48.v20220622

hardikp-fens commented 1 year ago
gharris1727 commented 1 year ago

@Hardik-Parikh @hardikp-fens Can you share more details about your installation? In particular:

Unfortunately the stacktrace you provided does not contain the error which caused the OpenSearchConnectorConfig class to fail to initialize. That class has a couple of external dependencies that might be missing or conflicting based on how it was installed.

If the connector JARs were added directly to the main kafka installation, there may be conflicting versions of the same dependency. Instead, the connector should be installed in a different location, and the plugin.path configured to point at the directory containing the directory containing the plugin jar. For example, if your jar is in /path/to/outer/inner/opensearch-connector-for-apache-kafka-3.0.0.jar, plugin.path should be set to /path/to/outer.

If the plugin.path is pointing at the connector jar directly, or pointing at the directory containing the plugin jar (/path/to/outer/inner), that would explain why the shadow (uber) jar is working, but the normal installation wasn't. In both of those cases, the normal installation won't find the dependency jars, as they aren't considered part of the same plugin.

I hope this helps, and lets you get back to the mainline release instead of the custom-built jar.