SensitiveQuestions / list

R list package for analyzing list experiments, also known as the item count technique
GNU General Public License v3.0
7 stars 5 forks source link

Errors in "ML model" routine #20

Closed itpradise closed 2 years ago

itpradise commented 2 years ago

Dears, I was running some routines of ictreg regarding the ML model with my data frame but it wasn't possible. I ran the routines of standard design ML, ceiling and floor effects alone and both. Let me show the error messages:

  1. Standard design ML:
Error in if (llik.const < pllik.const) warning("log-likelihood is not monotonically increasing.") : 
  missing value where TRUE/FALSE needed
In addition: Warning messages:
1: glm.fit: algorithm did not converge 
2: glm.fit: fitted probabilities numerically 0 or 1 occurred
  1. Ceiling effects:
Error in optim(par, obs.llik.binom.optim.boundary, method = "BFGS", J = J,  : 
  non-finite finite-difference value [1]
In addition: Warning messages:
1: glm.fit: fitted probabilities numerically 0 or 1 occurred 
2: In ictreg(resp ~ raca2 + escolaridade2 + city2 + parceiro2 + religiao2 +  :
  log-likelihood is not monotonically increasing.
  1. Floor effects:
Error in optim(par, obs.llik.binom.optim.boundary, method = "BFGS", J = J,  : 
  non-finite finite-difference value [7]
  1. Ceiling and floor effects:
Error in if (llik < pllik) warning("log-likelihood is not monotonically increasing.") : 
  missing value where TRUE/FALSE needed
In addition: Warning messages:
1: glm.fit: algorithm did not converge 
2: glm.fit: fitted probabilities numerically 0 or 1 occurred 

I managed run the others routines as Differences-in-means and NLS successfully. The problem is only with the ML model. Could anyone help me with this? I'll be very grateful! Thanks!

graemeblair commented 2 years ago

Have you run the design effects test using ict.test? Can you post the results? These results may be due to design effects or small numbers of observations in some cells of the Z, Y crosstab.

itpradise commented 2 years ago

Yes, I did. I'm working with J = 4 control statements.

This is the result of ict.test:

Test for List Experiment Design Effects

Estimated population proportions 
                          est.    s.e.
pi(Y_i(0) = 0, Z_i = 1) -0.0062 0.0083
pi(Y_i(0) = 1, Z_i = 1)  0.0001 0.0135
pi(Y_i(0) = 2, Z_i = 1)  0.0131 0.0141
pi(Y_i(0) = 3, Z_i = 1)  0.0141 0.0101
pi(Y_i(0) = 4, Z_i = 1)  0.0199 0.0029
pi(Y_i(0) = 0, Z_i = 0)  0.0918 0.0059
pi(Y_i(0) = 1, Z_i = 0)  0.2329 0.0112
pi(Y_i(0) = 2, Z_i = 0)  0.2867 0.0139
pi(Y_i(0) = 3, Z_i = 0)  0.2327 0.0124
pi(Y_i(0) = 4, Z_i = 0)  0.1149 0.0076

 Y_i(0) is the (latent) count of 'yes' responses to the control items. Z_i is the (latent) binary response to the sensitive item.

Bonferroni-corrected p-value
If this value is below alpha, you reject the null hypothesis of no design effect. If it is above alpha, you fail to reject the null.

Sensitive Item 1 
        0.720661

Could the problem of the small number of observations be associated with the presence of NA's? Because I tried to run the models without the variables that contain NA's and the error persisted.

graemeblair commented 2 years ago

The models can be fragile when there are small numbers of observations in some Y/Z cells. You can try to change the starting values, such as setting fit.start = "nls". If that doesn't work, send me an email with your anonymized data and I can take a look - graeme.blair@ucla.edu