ydataai / ydata-synthetic

Synthetic data generators for tabular and time-series data
https://docs.synthetic.ydata.ai
MIT License
1.45k stars 236 forks source link

[FEAT] NaNs are not supported in CTGAN model #346

Open shutsao opened 5 months ago

shutsao commented 5 months ago

Is your feature request related to a problem? Please describe. ValueError: Input X contains NaN. BayesianGaussianMixture does not accept missing values encoded as NaN natively.

Describe the solution you'd like However, missing values can be legitimate. For example, Patient A has a complex disease and is given 5 drugs, whereas Patient B doesn't have a complex disease and is given 1 drug. However, drug names are recorded across drug 1 to drug 5 columns and dosage is recorded across dosage 1 to dosage 5 columns. This leads to 8 missing values for Patient B but these missing values shouldn't be imputed because they are not applicable legitimately. Therefore, it may be more ideal to keep missing values as they are regardless of variable types.

Additional context The example described occurs frequently in health administrative data.