Closed dduong1603 closed 2 months ago
Hi, I am unable to reproduce. I actually tried your URL, and it works fine with 1.2.2
.
I changed the port to 2223
, and it then correctly failed.
Can you check that there are no conflicts in your env? Meaning, an env variable is not overriding your env.yaml config.
Also, run with DEBUG=TRUE
, like DEBUG=TRUE sling conns test sftp
to see more details.
Hi, sorry I wasn't specfic enough. sling conns test
works fine, but the command that failed for me was sling run
This works fine:
DEBUG=TRUE sling conns test TEST_SFTP
4:06PM DBG opened "sqlite" connection (conn-sqlite-BqY)
4:06PM DBG adding default private key (<redacted>/.ssh/id_rsa) as auth method for SFTP
4:06PM DBG file discover inputs: {"column_level":false,"pattern":"","recursive":false,"url":"sftp://demo:demo@demo.wftpserver.com:2222/"}
4:06PM DBG unfiltered nodes returned: 2
4:06PM DBG ["/upload/","/download/"]
4:06PM INF success!
But when I run:
sling run \
--src-conn WAREHOUSE \
--src-stream 'analytics.dim__dates' \
--tgt-conn TEST_SFTP \
--tgt-object '{stream_schema}/{stream_table}_{YYYY}_{MM}_{DD}.csv' \
--tgt-options '{"mode": "full-refresh", "format":"csv"}' -d
I got (this run happened on my Mac, but same thing with linux)
2024-04-09 16:09:35 DBG adding default private key (<redacted>/.ssh/id_rsa) as auth method for SFTP
2024-04-09 16:09:36 DBG Sling version: 1.2.3 (darwin arm64)
2024-04-09 16:09:36 DBG type is db-file
2024-04-09 16:09:36 DBG using source options: {"empty_as_null":true,"null_if":"NULL","datetime_format":"AUTO","max_decimals":-1}
2024-04-09 16:09:36 DBG using target options: {"header":true,"compression":"AUTO","concurrency":7,"datetime_format":"auto","delimiter":",","format":"csv","max_decimals":-1,"use_bulk":true,"add_new_columns":true,"column_casing":"source"}
2024-04-09 16:09:36 INF connecting to source database (postgres)
2024-04-09 16:09:36 DBG opened "postgres" connection (conn-postgres-bq4)
2024-04-09 16:09:37 INF reading from source database
2024-04-09 16:09:37 DBG select * from "analytics"."dim__dates"
2024-04-09 16:09:37 INF writing to target file system (sftp)
2024-04-09 16:09:37 DBG adding default private key (<redacted>/.ssh/id_rsa) as auth method for SFTP
2024-04-09 16:09:41 DBG closed "postgres" connection (conn-postgres-bq4)
2024-04-09 16:09:41 INF execution failed
fatal:
~ failure running task (see docs @ https://docs.slingdata.io/sling-cli)
--- task_run.go:101 func1 ---
--- task_run.go:300 runDbToFile ---
~ execution failed
--- task_run_write.go:39 WriteToFile ---
~ Could not WriteToFile
--- fs.go:147 NewFileSysClientFromURLContext ---
~ Could not obtain client for: sftp
--- fs.go:120 NewFileSysClientContext ---
~ Error initiating File Sys Client
--- fs_sftp.go:33 Init ---
~ unable to connect to ssh server
--- fs_sftp.go:107 Connect ---
~ unable to connect to ssh server demo.wftpserver.com:22
--- ssh.go:107 Connect ---
ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey password], no supported methods remain
Was able to reproduce, thanks for reporting.
Fixed here: https://github.com/slingdata-io/sling-cli/pull/253/commits/3ef5fd67ce9fd2c62faef07dd860a9f338a8e193 in v1.2.4
. Feel free to build & test.
Closing.
Issue Description
Description of the issue:
Sling version (
sling --version
):1.2.2
Operating System (
linux
,mac
,windows
):linux
Whenever I use an SFTP server with the default port 22
sling
works fine, but if I need to use a server with a non-default portsling
will fail with an error(in this case the demo server has an SFTP port of 2222, and I define the connection with
sling conns set SFTP url=sftp://demo:demo@demo.wftpserver.com:2222
)