[X] I had searched in the feature and found no similar feature requirement.
Description
When using sftp as a source, add an optional configuration option to only read the current specified directory, without recursively reading all files in that directory and its subdirectories; And when file_format_type is specified, only files with the suffix related to that type will be read.
Usage Scenario
No response
Related issues
When sftp is used as the source and file_format_type is Excel, the type of fields will not be automatically converted, which leads to type conversion errors when Excel data contains both int and string; In addition, empty areas in Excel cannot be read and should be automatically read as null values.
Search before asking
Description
When using sftp as a source, add an optional configuration option to only read the current specified directory, without recursively reading all files in that directory and its subdirectories; And when file_format_type is specified, only files with the suffix related to that type will be read.
Usage Scenario
No response
Related issues
When sftp is used as the source and file_format_type is Excel, the type of fields will not be automatically converted, which leads to type conversion errors when Excel data contains both int and string; In addition, empty areas in Excel cannot be read and should be automatically read as null values.
Are you willing to submit a PR?
Code of Conduct