sdv-dev / SDMetrics

Metrics to evaluate quality and efficacy of synthetic datasets.
https://docs.sdv.dev/sdmetrics
MIT License
201 stars 45 forks source link

Check that every property score are index-free #583

Closed R-Palazzo closed 1 month ago

R-Palazzo commented 3 months ago

Problem Description

A property score, like a metric, should not depend on the index of the real and synthetic data. It should only depend on their column values. It would be nice to have some integration tests that check that this is the case for all our current properties (single and multi-table) as well as for future ones.

Expected behavior

Write a simple integration test where real and synthetic data are the same and only differ by their indexes. Check that the property score is 1.0. Apply this test to all the properties (keeping in mind that it should work for future ones also). In the case of some property scores not being 1.0: