Add fake credit card data from scoofy example data into the glue job - Githubissues

kreuzwerker / kreuzlaker

11 stars 2 forks source link

Add fake credit card data from scoofy example data into the glue job #25

Open jankatins opened 1 year ago

jankatins commented 1 year ago

We added PII data so we can show how that is soposed to be handled. This is in the raw data, but not selecting it when doing the parquet conversion.

Tasks:

Figure out a hash functions for glue (ask @jankatins, he has some examples ...)
Add it to the glue job which transforms the scoofy data
Push the hash through in dbt?

DoD:

hashed data is available in at least one the final dataset (e.g. latest credit card hash)