search

kreuzwerker / kreuzlaker

11 stars 2 forks source link

Refactor main cdk stack into constructs #20

Open jankatins opened 1 year ago

jankatins commented 1 year ago

The main stack is too big, it needs splitting into multiple constructs

DoD

jankatins commented 1 year ago

Another thing to refactor is splitting the "data generation" (currently the copy job/scoofy data generation(#24), but potentially also the DB for cdc (#7)) and the data lake stuff and passing the raw data lake area to the data generation stack. That way the data lake stack is isolated what I think of as the "operational systems".

Not sure who is repsonsible to put data into the raw area: the source systems or the data lake. Probably the source system (e.g. that system defines the cdc target?)