k2v-academy / K2View-Academy

Other
0 stars 0 forks source link

TDM - what is right way to filter out / ignore BE entities if no data extracted for every table in all child LUs in BE #1047

Closed yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo closed 4 months ago

yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo commented 4 months ago

Hello, in one of customer scenarios for TDM we have BE hierarchy with 3 LUs: one parent and 2 child.

In parent LU we have 5 parameters, for child LUs - none.

They first ran 'Extract' task to bring parameters into TDM so later end-users can go and select parameters combination to bring data from source system.

They consider 'valid' entity only if it has data in all tables of both child LUs. In case at least one of tables in one of child LUs brings 0 records from source - they do not want this BE to be extracted or available via params selection for end users.

Since we have all parameters defined at parent LU level, even if we 'fail' get / sync for child LU based 0 records count for table - we still have parameters stored in TDM DB and they can lead to invalid BE extraction later.

What is the right way to filter those?


Best regards, Andrey

eHK5dcC00peRJzSmBkzpRSXRdBTJxvlzYiuW95S commented 4 months ago

Hi.

We cannot 'discard' parameters, but you can add a parameter of 'complete_child_data' to the child LU and set it to true only if all the child data exists for the entity. The task creator can select entities where the complete_child_data is true.

Tali