search

kearnz / autoimpute

Python package for Imputation Methods
MIT License
237 stars 19 forks source link

ValueError: Mass matrix contains zeros on the diagonal. The derivative of RV `alpha`.ravel()[0] is zero. #62

Closed orydatadudes closed 1 year ago

orydatadudes commented 3 years ago

i got the above error when trying to use autoimpute on data frame with two columns from kaggle india air quality data

df[['NO', 'NO2'].head() NO NO2 0 0.48 13.58 1 0.77 25.43 2 5.35 44.17 3 23.23 47.66 4 43.73 45.93

df[['NO', 'NO2']].dtypes: NO float64 NO2 float64

np.isnan(df[['NO', 'NO2']]).sum() NO 1948 NO2 836

i also try taking N rows from df so maybe i could find the rows that responsible to the problem: when i took top 10820 rows it work when i took took top 10825 it didn't work when i tool rows from 10810-10830 it work

what is the problem and what should i do thanks

Jaspernwd commented 3 years ago

Same problem!

kearnz commented 3 years ago

Hi @orydatadudes, @Jaspernwd - this issue can be quite difficult to debug, as it stems from the sampling process of the underlying pymc3 library. It generally has to do with how you initialize the underlying prior distribution when using bayesian methods. If you don't explicitly model the distribution yourself, autoimpute tries to do it for you, but that doesn't always work.