search

mansenfranzen / pywrangler

Advanced data wrangling for python
https://github.com/mansenfranzen/pywrangler
MIT License
11 stars 4 forks source link

Pyspark check for existing columns - ignore case sensitive #32

Closed mansenfranzen closed 4 years ago

mansenfranzen commented 4 years ago

Pyspark does not treat column names case sensitive by default. Therefore, checks for existing column names should ignore any case information.

See here: https://stackoverflow.com/questions/42946104/enable-case-sensitivity-for-spark-sql-globally

mansenfranzen commented 4 years ago

Closed with #33