Open shaikmanu797 opened 5 years ago
@samuel-pt, please review the PR and let me know if you think any additional changes should be made.
Ei @shaikmanu797 @samuel-pt Any forecast to have this feature merge to master? thank you 👍 !
@marcraminv @fernandomora @vejeta @sunayansaikia @ezra-at-lumedic
Since this PR has not been reviewed for more than a year now and the contributors in this repository seems to be inactive for a long time.
I had to go with my own implementation of the API to fix the issue. Feel free to take a look at the package and any feedback is highly appreciated.
Reading the coalesce numPartitions value from custom SparkSession config key spark.sftp.coalesce.partitions, default numPartitions set to 1
Test run log attached: test.log