search

apache / camel-kafka-connector

Camel Kafka Connector allows you to use all Camel components as Kafka Connect connectors
https://camel.apache.org
Apache License 2.0
152 stars 100 forks source link

SFTP Connector throwing Auth Fail error while connecting via PrivateKey #1573

Closed Shubham-Garg20 closed 3 weeks ago

Shubham-Garg20 commented 11 months ago

We're getting the following error when we try to reach out to our local SFTP server via PrivateKeys:

[2023-11-02 12:47:30,299] WARN [CamelSftpSourceConnector|task-0] Error auto creating directory: demos/ due Cannot connect to sftp://root@localhost:24. This exception is ignored. (org.apache.camel.component.file.remote.SftpConsumer:72)
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot connect to sftp://demo@localhost:24
    at org.apache.camel.component.file.remote.SftpOperations.connect(SftpOperations.java:157)
    at org.apache.camel.component.file.remote.RemoteFileConsumer.connectIfNecessary(RemoteFileConsumer.java:236)
    at org.apache.camel.component.file.remote.SftpConsumer.doStart(SftpConsumer.java:68)
    at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
    at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
    at org.apache.camel.impl.engine.AbstractCamelContext.startService(AbstractCamelContext.java:3464)
    at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRouteConsumers(InternalRouteStartupManager.java:401)
    at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartRouteConsumers(InternalRouteStartupManager.java:319)
    at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:213)
    at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:147)
    at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3167)
    at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2847)
    at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2798)
    at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
    at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2494)
    at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:245)
    at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
    at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
    at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:172)
    at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:274)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
    at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:75)
    at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:181)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.jcraft.jsch.JSchException: Auth fail
    at com.jcraft.jsch.Session.connect(Session.java:519)
    at org.apache.camel.component.file.remote.SftpOperations.connect(SftpOperations.java:127)
    ... 28 more

Here is the config:

name=CamelSftpSourceConnector
connector.class=org.apache.camel.kafkaconnector.sftp.CamelSftpSourceConnector
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
transforms=RemoteTransformer
transforms.RemoteTransformer.type=org.apache.camel.kafkaconnector.RemoteFileTransforms

topics=mytopic

camel.source.path.host=localhost
camel.source.path.port=24
camel.source.path.directoryName=demos/
camel.source.endpoint.recursive=true
camel.source.endpoint.username=root
camel.kamelet.sftp-source.privateKeyFile=<link to privatekey file>
camel.source.endpoint.noop=false
camel.source.endpoint.move=.done

Although, when I'm trying to connect to the same SFTP server via the external tools like FileZilla, it successfully logs me in with the same private key and user. Can you please help us resolve the issue?

oscerd commented 10 months ago

What is the version used? Also how are you specify the link to private key file?

Shubham-Garg20 commented 10 months ago

What is the version used? Also how are you specify the link to private key file?

We have the private key stored locally in the laptop, and we are giving the path to the same. We are using the latest version of connector.

oscerd commented 10 months ago

The version of camel-kafka-connector I meant to ask.

Also you need to prepend file:// if you use a local file

Shubham-Garg20 commented 10 months ago

We're using 0.11.5 version of camel-kafka-connector. And the way we're setting privatekeyfile is: camel.kamelet.sftp-source.privateKeyFile=/Users/test/Downloads/test.ppk

oscerd commented 10 months ago

try with file:////Users/test/Downloads/test.ppk

oscerd commented 3 weeks ago

No feedback, closing.