Open clojurians-org opened 4 years ago
@binarylogic @jszwedko Any updates on this? SFTP via a single or pool of ssh connections would be very useful!
This isn't on our near-term roadmap at the moment. We've recently started trying out integrations through OpenDAL, and while not supported yet sftp is on their roadmap.
@spencergilbert Thanks for the info. Looking forward to all this in the future!
@spencergilbert Although not released yet, sftp support was recently merged to OpenDAL :-)
SFTP support has been released by OpenDAL: https://github.com/apache/incubator-opendal/releases/tag/v0.33.3
Maybe we can start by adding sftp as a sink.
@Xuanwo That would be awesome!
Has there been any updates on this? 🤔
Has there been any updates on this? 🤔
Not that I'm aware of. I think a sink leveraging OpenDAL would be a good start though. If anyone is so motivated you can see the WebHDFS sink for an example of what this might look like (it also uses OpenDAL).
Hi, I start a draft at https://github.com/vectordotdev/vector/pull/18076. Welcome to have a look!
Adding new opendal-based sinks is quite simple, as it mostly involves configuration and documentation tasks.
Hi, I start a draft at #18076. Welcome to have a look!
Adding new opendal-based sinks is quite simple, as it mostly involves configuration and documentation tasks.
Looking forward to this!
@Xuanwo Any updates on your PR? It has been stale for a few months now
we have no access right to install vector and mount disk for remote data source. so we can only pull data from remote by sftp.
kafka connect already has a sftp connect.
https://docs.confluent.io/current/connect/kafka-connect-sftp/index.html