Fix data source writer option from codec to compression - Githubissues

data-derp / small-exercises

6 stars 9 forks source link

Fix data source writer option from codec to compression #6

Closed yulian-tw closed 2 years ago

yulian-tw commented 2 years ago

Changes in wrangling-in-spark folder:

The fix and added references to datasource options for 12: DataFrameWriter exercise
Deleted manifests in wrangling-in-spark
Add CONTRIBUTIONS.md to indicate atomic commits to .dbc, .zip and the unzipped notebooks.

Also in the dbc and zip

04: SparkContext and Simple RDDs.py : Add a step to use .glom() to illustrate how RDD works
02: DataFrameReader.py : Format in DataFrameReader for JSON file schema to use pre tag instead of h6 in md
03: Spark Functions.py : https://github.com/data-derp/small-exercises/pull/5