filter out additional files from temp file location for Databricks

springml / spark-sftp

Spark connector for SFTP

Apache License 2.0

100 stars 98 forks source link

filter out additional files from temp file location for Databricks #62

Closed sshah90 closed 5 years ago

sshah90 commented 5 years ago

In Databricks, whenever write operation triggered it saves committed and started file which is not filtered out in copiedFile function that's why even you give tempLocation explicitly it's not able to pick up a csv file.