search

awslabs / kinesis-kafka-connector

kinesis-kafka-connector is connector based on Kafka Connect to publish messages to Amazon Kinesis streams or Amazon Kinesis Firehose.
Apache License 2.0
153 stars 91 forks source link

connect-standalone error: connector stopped caused by "Unsupported major.minor version" #6

Closed guoxun19 closed 7 years ago

guoxun19 commented 7 years ago

Hi,

I met the error when run "connect-standalone.sh", the error log is as below. Could you please help me? Thanks a lot! Kafka: 0.10.0.0 OS: CentOS 6.5

[2017-09-14 05:37:10,178] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:100) java.lang.UnsupportedClassVersionError: com/amazon/kinesis/kafka/AmazonKinesisSinkConnector : Unsupported major.minor v at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) at java.net.URLClassLoader.access$100(URLClassLoader.java:64) at java.net.URLClassLoader$1.run(URLClassLoader.java:354) at java.net.URLClassLoader$1.run(URLClassLoader.java:348) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:347) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:195) at org.apache.kafka.connect.runtime.Worker.getConnectorClass(Worker.java:199) at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:166) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:250) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:164) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:94) [2017-09-14 05:37:10,186] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:68) [2017-09-14 05:37:10,186] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:154) [2017-09-14 05:37:10,199] INFO Stopped ServerConnector@34c0328{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.Server06) [2017-09-14 05:37:10,250] INFO Stopped o.e.j.s.ServletContextHandler@1aa42f23{/,null,UNAVAILABLE} (org.eclipse.jetty.ser.ContextHandler:865) [2017-09-14 05:37:10,259] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:165) [2017-09-14 05:37:10,259] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:77) [2017-09-14 05:37:10,259] INFO Stopping connector kinesiskafkaConnector (org.apache.kafka.connect.runtime.Worker:278) [2017-09-14 05:37:10,260] ERROR Error shutting down connector kinesiskafkaConnector: (org.apache.kafka.connect.runtime.StandaloneHerder:87) org.apache.kafka.connect.errors.ConnectException: Connector kinesiskafkaConnector not found in this worker. at org.apache.kafka.connect.runtime.Worker.stopConnector(Worker.java:282) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.stop(StandaloneHerder.java:85) at org.apache.kafka.connect.runtime.Connect.stop(Connect.java:71) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:101) [2017-09-14 05:37:10,260] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:128) [2017-09-14 05:37:10,260] WARN Shutting down tasks [] uncleanly; herder should have shut down tasks before the Worker i(org.apache.kafka.connect.runtime.Worker:141) [2017-09-14 05:37:10,260] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore: [2017-09-14 05:37:10,260] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:151) [2017-09-14 05:37:13,116] INFO Reflections took 5223 ms to scan 63 urls, producing 3845 keys and 27838 values (org.refflections:229) [2017-09-14 05:37:13,125] WARN could not get type for name com.amazon.kinesis.kafka.AmazonKinesisSinkConnector from anyer (org.reflections.Reflections:384) org.reflections.ReflectionsException: could not get type for name com.amazon.kinesis.kafka.AmazonKinesisSinkConnector at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:378) at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:397) at org.reflections.Reflections.getSubTypesOf(Reflections.java:367) at org.apache.kafka.connect.runtime.AbstractHerder.connectorPlugins(AbstractHerder.java:270) at org.apache.kafka.connect.runtime.AbstractHerder$1.run(AbstractHerder.java:377) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.UnsupportedClassVersionError: com/amazon/kinesis/kafka/AmazonKinesisSinkConnector : Unsupported maersion 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) at java.net.URLClassLoader.access$100(URLClassLoader.java:64) at java.net.URLClassLoader$1.run(URLClassLoader.java:354) at java.net.URLClassLoader$1.run(URLClassLoader.java:348) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:347) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:376)

nehalmehta commented 7 years ago

Hi @guoxun19 Which JAVA version are you using to build the artifact? And which JAVA Version are you using to run Kafka - Connect?

guoxun19 commented 7 years ago

Hi, @nehalmehta , java version to build artifact: 1.8 ; java version to run connect script: 1.7 Do I need to use the same java version to build and run? Thank you!

nehalmehta commented 7 years ago

@guoxun19 - Yeah. JRE has to be same or higher version.

guoxun19 commented 7 years ago

@nehalmehta thank you~ I update java version for running env to 1.8 and the issue has been resolved.

I have another question about credentials, I run connect script as Root, I put AWS credentials file under /root/.aws/credentials, but it doesn't work, the script "Unable to load AWS credentials from any provider in the chain", where should I put the credentials file and what the context format should it have?

nehalmehta commented 7 years ago

You should be able to configure as per here - http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html#credentials-default

Look at 3# in it and it should be under ~/.aws/credentials (depends on OS). I am closing this issue but if you have questions about credentials which are not answered in above link please open another issue.

Thanks, Nehal