MBB-team / VBA-toolbox

The VBA toolbox
GNU General Public License v3.0
129 stars 67 forks source link

IsYout is not taken into account in VBA_Iphi #19

Closed lionel-rigoux closed 5 years ago

lionel-rigoux commented 8 years ago

The computation og the error term dy2 in VBA_Iphi does not exclude the points that should be excluded according to isYout. This causes bugs when NaNs are present in the data and, more problematically, can silently bias the inference. VBA_Iphi_extended does not have the problem.

jdaunize commented 8 years ago

Lionel,

Normally, there is a bit of code in VBA_check that ensures that the data precision matrices priors.iQy{t} are zeroed at the corresponding data points:

% ensure excluded data consistency if ~options.binomial for t=1:dim.nt diQ = diag(priors.iQy{t}).~options.isYout(:,t); options.isYout(:,t) = ~diQ; priors.iQy{t} = diag(diQ)_priors.iQy{t}*diag(diQ); end end

In turn, this ensures that dy2 is removing the excluded data points (cf., in VBA_IPhi): dy2 = dy2 + dy(:,t)'_iQy{t}_dy(:,t);

Are you sure of what you are saying? Jean.

lionel-rigoux commented 8 years ago

The problem is that 0*NaN gives a NaN, so if the data are effectively excluded, a NaN in the data will generate a nan dy2 even if it is excluded according to isYout. So I retract my dramatic statement about potential silent problems, but I am sure that NaNs break the inversion!