search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.29k stars 4.15k forks source link

SFTP Destination #4875

Open mystmaker33 opened 3 years ago

mystmaker33 commented 3 years ago

Tell us about the new connector you’d like to have

I would like an SFTP destination for csvs based on tables in my database.

Describe the context around this new connector

We often need to regularly output csv files for our vendors with our data in them. Many of them use SFTPs to receive the csvs.

Describe the alternative you are considering or using

Right now I have scheduled python scripts that query our database, write the csv and post it to the desired SFTP

Are you willing to submit a PR?

Maybe! I can take a look, but don't want to guarantee anything :)

snikhil24 commented 2 years ago

Hello! Would like to know if there's any update on this? I'm also looking to use SFTP as a destination. @mystmaker33 did you find some solution for this? Thanks in advance!

haithem-souala commented 2 years ago

Hey @snikhil24, @mystmaker33 I started working on this one, if you could, your contribution will be much appreciated! https://github.com/airbytehq/airbyte/pull/19536

natikgadzhi commented 1 month ago

Hey! I'm cleaning up issues, and want to clarify. So we have destination-sftp-json, and you'd like to have destination-sftp-csv, is that it? Would it be a better fit to have destination-sftp with configurable format instead?

mystmaker33 commented 1 month ago

Personally I think a configurable destination format makes the most sense, as different vendors need their files formatted differently.

On Sun, Oct 13, 2024, 8:11 PM Natik Gadzhi @.***> wrote:

Hey! I'm cleaning up issues, and want to clarify. So we have destination-sftp-json, and you'd like to have destination-sftp-csv, is that it? Would it be a better fit to have destination-sftp with configurable format instead?

— Reply to this email directly, view it on GitHub https://github.com/airbytehq/airbyte/issues/4875#issuecomment-2409434235, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHJLVMVOTBYOCODZ7YGLP3Z3MD4LAVCNFSM6AAAAABP35JTN2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBZGQZTIMRTGU . You are receiving this because you were mentioned.Message ID: @.***>