ydataai / ydata-synthetic

Synthetic data generators for tabular and time-series data
https://docs.synthetic.ydata.ai
MIT License
1.38k stars 232 forks source link

[FEAT] NaNs are not supported in CTGAN model #346

Open shutsao opened 1 month ago

shutsao commented 1 month ago

Is your feature request related to a problem? Please describe. ValueError: Input X contains NaN. BayesianGaussianMixture does not accept missing values encoded as NaN natively.

Describe the solution you'd like However, missing values can be legitimate. For example, Patient A has a complex disease and is given 5 drugs, whereas Patient B doesn't have a complex disease and is given 1 drug. However, drug names are recorded across drug 1 to drug 5 columns and dosage is recorded across dosage 1 to dosage 5 columns. This leads to 8 missing values for Patient B but these missing values shouldn't be imputed because they are not applicable legitimately. Therefore, it may be more ideal to keep missing values as they are regardless of variable types.

Additional context The example described occurs frequently in health administrative data.