Open adityashekhar146 opened 3 years ago
@adityashekhar146 take a look at https://github.com/arcizon/spark-filetransfer repository (supports Spark 3) which I had developed if it helps to solve your issue.
@shaikmanu797 your spark-filetransfer has issues. I tried it with spark 3 and i am seeing file content got changed after sending it into SFTP server
@adityashekhar146 Currently, the spark-sftp package doesn't work when trying to use in Spark 3.0. Are there any plans to extend the support for Spark 3? We are waiting for spark 3 Support .
We are also looking for same sftp connector for spark3.x, so is there any plan to have this soon.
Why there is no commit or improvement on this repo. I also see scala 2.12 pull request is open from long time too.
@adityashekhar146 take a look at https://github.com/arcizon/spark-filetransfer repository (supports Spark 3) which I had developed if it helps to solve your issue.
This connector library perfectly works in my case. (Spark 3.1.2, Scala 2.12), @shaikmanu797 thanks for great work.
@adityashekhar146 are there any plans for provide support for scala 2.13
Currently, the spark-sftp package doesn't work when trying to use in Spark 3.0. Are there any plans to extend the support for Spark 3?