Normalize resource - Githubissues

datahq / dataflows

DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.

https://dataflows.org

MIT License

198 stars 40 forks source link

Open akariv opened 5 years ago

akariv commented 5 years ago

Add ability to normalize a resource - i.e.

extract a few columns out of a resource into a deduplicated 'lookup table' resource
add proper cross indexes and pointers (i.e. replace extracted columns with the index of the
create foreign key relationships between the two resources.

rufuspollock commented 5 years ago

@akariv can you give a bit more detail - i normally think of "normalize" as like "unpivot" cf #72

akariv commented 5 years ago

Yes - unpivoting is actually a process for denormalizing data.

Normalizing is the process of removing duplication, as can be read about here: https://en.wikipedia.org/wiki/Database_normalization

rufuspollock commented 5 years ago

Got it and clear. Thanks :smile: