Open sjrusso8 opened 2 weeks ago
Implement the initial methods to read and write .csv, .json, .orc, .parquet, and .text.
.csv
.json
.orc
.parquet
.text
Consider creating ConfigOpts trait for each of those file options and have a custom struct represent the options for each of those file types.
ConfigOpts
Create the Options and modify the opts object. The object is passed into the method.
let mut opts = CsvOptions::new() opts.header = true; opts.delimiter = b'|'; let df = spark.read().csv(path, opts)
Example of what the function signature might look like
impl DataFrameReader { .... pub fn csv<C: ConfigOpts>(path: &str, opts: Some(C)) }
Description
Implement the initial methods to read and write
.csv
,.json
,.orc
,.parquet
, and.text
.Consider creating
ConfigOpts
trait for each of those file options and have a custom struct represent the options for each of those file types.Example with CSV
Create the Options and modify the opts object. The object is passed into the method.
Example of what the function signature might look like