search

aliyun / kafka-connect-oss

Kafka Connect suite of connectors for OSS
Other
29 stars 13 forks source link

No FileSystem for scheme "oss" #4

Open ChenShuai1981 opened 4 years ago

ChenShuai1981 commented 4 years ago

I am going to persistent kafka messages to aliyun oss using confluent suite, I put my compiled kafka-connect-oss-5.5.1.jar in $CONFLUENT_HOME/share/java/kafka-connect-oss folder and create a new connector, then I found the following exceptions in kafka connect, what does it mean? How to fix it? Thanks!

org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.ConnectException: java.lang.reflect.InvocationTargetException at com.aliyun.oss.connect.kafka.OSSSinkTask.start(OSSSinkTask.java:112) at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:304) at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:195) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kafka.connect.errors.ConnectException: java.lang.reflect.InvocationTargetException at io.confluent.connect.storage.StorageFactory.createStorage(StorageFactory.java:55) at com.aliyun.oss.connect.kafka.OSSSinkTask.start(OSSSinkTask.java:104) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at io.confluent.connect.storage.StorageFactory.createStorage(StorageFactory.java:50) ... 10 more Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "oss" at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3390) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3411) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3474) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3442) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524) at com.aliyun.oss.connect.kafka.storage.OSSStorage.(OSSStorage.java:50) ... 15 more

wujinhu commented 3 years ago

It seems your configuration is wrong, please refer the documentation.

liyichao1872 commented 1 year ago

没有引用 core-site.xml ,把这个放到 kafka的配置文件中，还有src/main/resources/ 中，重新 mvn install 一下

viscory commented 1 year ago

Error is still there. I added core-site.xml to src/main/resources and re-ran mvn clean install. It did not work. This is my core.xml: https://gist.github.com/viscory/de9c9a581f781157b3b0b663a51ffebd

Please provide guidance.