Closed SSchotten closed 3 months ago
An attempt at clarifying the workings of df_check.py. Main changes include:
df_check.py
df_check
run
ValidationSettings
common.py
README.md
spark_session
dq_rules_dict
Should we rename df_check.py to something more descriptive?
An attempt at clarifying the workings of
df_check.py
. Main changes include:df_check
-function into many smaller functions: entrypoint now is therun
-functionValidationSettings
-dataclass for storing all settings related to the validation of a single dataframecommon.py
for shared classes and functionsREADME.md
spark_session
anddq_rules_dict
)