Closed eemhu closed 4 months ago
internal pull request submitted for hdfs save/load CSV format support.
Should add documentation and support for adding files manually to HDFS and allow loading them with HDFS load, same for saving with HDFS save and using them without HDFS load.
Also add HEADER=BOOL parameter for load and save to allow csv header configuration. Allow providing schema in hdfs load. If HEADER=FALSE and no schema provided, load all to _raw column.
Internal PR up for review
Description
Allow saving to HDFS in CSV format.
Use case or motivation behind the feature request
Can be used to save teragrep results in a standard format.
Related issues
https://github.com/teragrep/pth_10/issues/132
Additional context
JSON also could be added, but CSV seems to be the higher priority for now. Loading should also be supported via hdfs load.