Statistical similarity - Chi-squared and Kolmogorov-Smirnov Tests

[garrgravarr]

Compare p-values of tests on input vs synthetic data.

[garrgravarr]

commit 24868683b7b31439654ff9b77e92853e3e4b99d8

Added subdirectory metrics.

Added file univariate_statistics.py

Functions

• kstest(real_data, synthetic_data) returns a dict mapping numerical variables to the Kolmogorov-Smirnov p-value of the comparison between real and synthetic data.
• cstest(real_data, synthetic_data) returns a dict mapping categorical variables to the Chi-square p-value for the comparison of the distributions of real and synthetic data. The data is first normalised so that the distributions are comparable.
• cstest_sdv(.,.) returns the predefined Chi-square metric of SDV.

ToDo: calculate a single score from all variable's comparison values. This could be proportional to the maximum norm.

[garrgravarr]

The infrastructure is there for custom metrics to be averaged to one score, so in principle this is done.

The default scoring in in the ASyH.Application pipeline is done with the SDMetrics' QualityScore, though.