ray-project / deltacat

A portable Pythonic Data Catalog API powered by Ray that brings exabyte-level scalability and fast, ACID-compliant, change-data-capture to your big data workloads.
Apache License 2.0
166 stars 23 forks source link

Minor Syntax Error in delta.py #338

Closed akindu-amazon closed 3 months ago

akindu-amazon commented 3 months ago

The len(tables) == len(annotations) assertion in _read_delta_file_envelopes (https://github.com/ray-project/deltacat/blob/main/deltacat/compute/compactor_v2/utils/delta.py#L45) seems to have a syntax error which allows the assertion to always pass.

assert (
    len(tables) == len(annotations),
    f"Unexpected Error: Length of downloaded delta manifest tables "
    f"({len(tables)}) doesn't match the length of delta manifest "
    f"annotations ({len(annotations)}).",
)

should be changed to something like

assert len(tables) == len(annotations), (
    f"Unexpected Error: Length of downloaded delta manifest tables "
    f"({len(tables)}) doesn't match the length of delta manifest "
    f"annotations ({len(annotations)})."
)

As a note, this same assertion is also used in compactor V1 (https://github.com/ray-project/deltacat/blob/main/deltacat/compute/compactor/steps/hash_bucket.py#L160)