Problems with too little variation

[reuning]

I'm trying to run a model with a stepwise time covariate. I'm curious about how a parameter changes before and after a certain time period. I estimated the model fine but I get the combined warning/error when trying to call it:

Estimates and 95% confidence intervals:
Error in boot::boot.ci(object@boot, conf = level, type = type, index = x) : 
  't' must of length 500
In addition: Warning message:
In .local(object, parm, level, ...) :
  Too little variation in the model. 174 replications (34.8%) are dropped from CI estimation

I'm not sure if this is a bug or a feature. In that, I don't know how concerned I should be with there being an issue with too little variation, as in, does this reflect a problematic model or just a problem in sampling.

[leifeld]

This means there are not enough time points or there is not enough variation between the time points that are included in order to evaluate the mean causal effect. Keep in mind that temporal effects using timecov have no variation between dyads within a time point. They only vary between time points. So you need sufficient numbers of time points for this to work in a meaningful way with bootstrapping. It's akin to the problem of multicollinearity in regression modelling. As an alternative, if you have only a few time points, you could try using mtergm instead of btergm.

[reuning]

Thanks for the quick reply. Right now I only have a handful of timepoints so that makes sense especially now that I think about what is bootstrapping here.

If you don't mind, I wanted to check to make sure I am including these covariates correctly. As I said, I am trying to estimate a model where I am seeing the difference in absdiff() before and after a timepoint. I created a list of matrices dim1 which contains the change statistics from absdiff() and then put it in as: timecov(dim1, maximum=5, transform=function(t) 1) + timecov(dim1, minimum=6, transform=function(t) 1) Is this the correct way of including it in the model? Thank you again for any help.

[leifeld]

That may actually explain the error message because there may be a collinearity between the first and the second term. You only need to include the second term with minimum = 6. This means a list of matrices is included where the first five time points are 0 and the remaining ones 1, and an interaction effect between that and your absdiff term is included. Additionally, you also need to include the absdiff list manually as a main effect using an edgecov model term for correct interpretability.

PS: just to be clear, do not include the first term.

[reuning]

Oh okay. I get it now. The timecov() terms are somewhat like interaction terms. This makes more sense. Thanks again for the help.

[leifeld]

They are interaction effects if you specify the x argument (default is NULL). If not, only the time main effect is included. Good luck!

[leifeld]

Oh wait, I think I made a mistake there. If you use this as an interaction effect, you probably need to include it a second time without the x argument for the main effect. The other things I wrote should be correct.

[reuning]

Wait, sorry let me just make sure I am on the same page. So what I am now estimating is:

btergm(networks~edges + nodecov("dim1") + 
                nodecov("dim2") + 
                edgecov(dim1) +
                timecov(dim1, minimum=6, transform=function(t) 1)) 
                edgecov(dim2) +
                timecov(dim2, minimum=6, transform=function(t) 1) +
                memory("innovation") +gwesp(decay=1, fixed=T) + 
                gwdsp(decay=1, fixed=T) + gwdegree(decay=1, fixed=T), verbose = T)

Where dim1 and dim2 are the change statistics from absdiff("dim1") and absdiff("dim2").

These are placements of the nodes in a dimensional space (generated separately from the network) and I have a theory that after time=5, the distance in the dimensional space won't matter as much and edges will form between those who were further away (put another way the coefficient on absdiff() which is negative will attenuate).

Is this correct?

[leifeld]

I think it should look like this, but you may want to double-check if this works:

btergm(networks ~ edges + 
            nodecov("dim1") + 
            nodecov("dim2") + 
            edgecov(dim1) +
            timecov(minimum = 6, transform = function(t) 1) + 
            timecov(dim1, minimum = 6, transform = function(t) 1) + 
            edgecov(dim2) +
            timecov(minimum = 6, transform = function(t) 1) +
            timecov(dim2, minimum = 6, transform = function(t) 1) +
            memory("innovation") +
            gwesp(decay = 1, fixed = TRUE) + 
            gwdsp(decay = 1, fixed = TRUE) + 
            gwdegree(decay = 1, fixed = TRUE), 
            verbose = TRUE)

Note the changed details at the end of your first timecov term and the two addition timecov main effects.

[reuning]

Hey thanks again for this. Sorry I didn't reply faster. I think I got it working and actually changed it a bit so I am not using a stepwise estimation. If I include two timecov() as you did in the last reply I get an issue which is I think the result of collinearity as that is basically including the same term twice. I ended up setting it up like this:

                timecov(transform=function(t) t) + 
                edgecov(dim1) +
                timecov(dim1, transform=function(t) t) + 
                edgecov(dim2) +
                timecov(dim2, transform=function(t) t)