Implement unit tests for MPLE existence check (or pigyback on existing ones).

[krivit]

We need to have some ongoing tests to make sure it triggers the warning and the right time and doesn't trigger it at the wrong time. @schmid86 , do you have any standard test cases you use?

[schmid86]

For the checks I used networks that I simulated for a current project I'm working on with Dave. Both of these networks have the same statistics, but for one the MPLE does exist and for the other it does not. Here is the code to generate these networks:

set.seed(123123)
#starting matrix
A <- matrix(rbinom(81, 1,0.3), 9,9)\
diag(A)<- 0
A <- as.network.matrix(A, directed=FALSE)
set.seed(777) 
san.net <- san(A ~ edges+triangles, target.stats=c(26,25))
m1 <- ergm(san.net ~ edges + triangles, estimate="MPLE") # MPLE does not exist!

Here is a similar network (same number of nodes, same statistics), where the MPLE does exist:

set.seed(888899292) 
san.net2 <- san(A ~ edges+triangles, target.stats=c(26,25))
m2 <- ergm(san.net2 ~ edges + triangles, estimate="MPLE") 

[drh20drh20]

Thanks, Christian. These examples are good ones and could be added to the tests directory somehow. I'm a bit concerned that if the san algorithm changes then the same seeds will not necessarily produce the same networks in the future. Thus, I'd recommend constructing the networks explicitly. For example, here's a trick one can use:

> m <- as.matrix(network(4, directed=FALSE)) # m represents our desired network object as an adjacency matrix
> cat(deparse(m), "\n")
structure(c(0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0), .Dim = c(4L,  4L), .Dimnames = list(c("1", "2", "3", "4"), c("1", "2", "3",  "4"))) 

Then we can insert the text obtained into the test and we've got the network desired without having to rely on anything stochastic:

newm <- 
structure(c(0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0), .Dim = c(4L,  4L), .Dimnames = list(c("1", "2", "3", "4"), c("1", "2", "3",  "4"))) 
nw <- network(newm, directed=FALSE)

[drh20drh20]

@krivit Christian and I have been discussing these tests and a question we both have is: How should an ergm object indicate that there is not an MPLE? In the example above, the coefficients clearly indicate to anyone who's done this that there is a problem, but it would be good to have some unambiguous way to indicate in the ergm object that no MPLE exists. What would you recommend?

[krivit]

I suggest using testthat and expect_warning(). This way, you can specify exactly what kind of warning (if any) the call should return.

[drh20drh20]

Thanks @krivit but aren't testthat and expect_warning only relevant in the context of running tests? We're talking about some kind of flag that ergm itself would set to indicate that there is no MPLE. Sorry if I'm misinterpreting your response.

[krivit]

Oh, sorry, I was thinking in the context of tests---to make sure the code fires a warning when it should and doesn't when it shouldn't. That's what this ticket was about.

Are you looking for a way to inform the user when printing a summary() message?

[drh20drh20]

Indeed, this issue came about due to the tests; sorry to expand the scope somewhat in this discussion.

But it struck us that it would be good practice for ergm to set some kind of flag in the event of a non-existent MPLE. Of course we could simply throw a warning at the time of ergm's MPLE fitting procedure, but we felt it would be good if there were, in addition, some permanent record in the ergm object that this happened. One possibility is to add yet another named item to the ergm-object list, say, MPLEexists. But we didn't want to further muck up the ergm object without bouncing this off you first. We thought you might have a good idea about how to code a non-existent MPLE and/or preferences about how NOT to do it.

[krivit]

It comes down to what you ultimately want to do with the information and where you want to do it, I think. What's the use case?

[drh20drh20]

For instance, the warning should reappear whenever someone uses summary.ergm on an ergm object whose MPLE does not exist. This won't happen if we merely throw the error at the time of fitting.