GoogleCloudPlatform / professional-services-data-validator

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match
Apache License 2.0
408 stars 119 forks source link

code cleanup around `get_source_ibis_table` and `get_target_ibis_table` in config_manager.py #1294

Open sundar-mudupalli-work opened 1 month ago

sundar-mudupalli-work commented 1 month ago

Hi,

There is some confusing logic and pre-conditions around these functions. It would be good to have a thorough review and cleanup this code. Specifically this line says:

        if self.validation_type == consts.CUSTOM_QUERY:
            self._target_ibis_table = clients.get_ibis_table(
                self.target_client, self.target_schema, self.target_table
            )

When you have a custom query - you don't have a target schema and a target table. So this code will most likely fail. It appears that this code is never executed. To make things more interesting _target_ibis_table does not appear to be set anywhere or in other places - so not sure how this would work. Same or similar comments on _source_ibis_table.

Things may have evolved since this code was written and this code is no longer needed and could use cleanup. Needs someone with a fresh perspective to review this and cleanup.

Sundar Mudupalli