statnet / ergm

Fit, Simulate and Diagnose Exponential-Family Models for Networks
Other
97 stars 37 forks source link

Error in estimation with ERGM 3.9.4 and tergm 3.5.2 #36

Closed dth2 closed 5 years ago

dth2 commented 5 years ago

I am fitting an ergm and I get the following error:

Error in UseMethod("as.edgelist") : no applicable method for 'as.edgelist' applied to an object of class "NULL" Error: $ operator is invalid for atomic vectors

The formula for the ergm is:

Formulas

formation.asmm <- ~edges + nodefactor("riskg",base = 3)+ nodefactor("yamsm", base=1) + absdiff("sqrt.age") + offset(nodefactor("oamsm",base = 1)) + offset(nodematch("yamsm", diff = TRUE, keep = 2)) + offset(nodematch("role.class", diff = TRUE, keep = 1:2)) + offset(nodefactor("debuted",base=2))

If I remove either the nodefactor("riskg", base =3) or the absdiff("sqrt.age") the model will converge. I can get the model to run as is by reverting to ergm_3.8.0 and tergm_3.4.1

sgoodreau commented 5 years ago

I started getting the same error when I updated to the latest packages on Oct 30, and found as a stop-gap measure for the work I was doing to reduce the population size. My guess is that it all has to do with the number of dyads in the MPLE estimation, which would be consistent with removing those specific model terms or reducing pop size both working. My code is all deeply integrated with evonet so hard for me to create an stand-alone example. Deven - are you able to do so?

dth2 commented 5 years ago

I can try to put together an example. This issue is a growing problem since I can no longer run Camp and SHAMP work with the same versions of ERGM and TERGM due to other dependencies.

krivit commented 5 years ago

Yes, please give me a reproducible example. Also, if possible, please try it with ergm and tergm from the master branch.

dth2 commented 5 years ago

The attached script recreates the problem. I also tried using the git versions of ERGM and TERGM (included in the attached script) and that did not work either, although the error was different.
ERGM TERM version comparison.txt

sgoodreau commented 5 years ago

OK I just spent some time diving deep into this. It seems to involve some pretty large, deep changes in ergm, so we definitely need @krivit to be involved. And my understanding is that all of @dth2 's modeling work on multiple projects for both @martinamorris and me has come to a halt as a result, so any time you have for it, @krivit, is vitally appreciated.

The issue appears during the process of pseudo-likelihood estimation, in the stack when ergm.mple calls ergm.pl.

In older versions of ergm, both ergm.pl and ergm.mple had Clist as their first argument, and it appears that this contains elements heads and tails, which represent the edges in the network.

In the new version of ergm (3.9.4), ergm.pl and ergm.mple do not have a function argument Clist. Instead, Clist is created inside ergm.pl, with the call Clist <- ergm.Cprepare(nw, m) on line 4.

For our example, this returns a list with many elements, none of which are called heads or tails. However, line 37 of ergm.pl contains the command:

el <- as.edgelist(cbind(Clist$tails, Clist$heads), n, directed = TRUE, bipartite = FALSE, loops = TRUE)

Since Clist$tails and Clist$heads are both NULL, this generates the error we've been seeing.

Maybe the reason these are null is because the network is empty. But we need to be able to pass empty networks here (and have always been able to, so if that's the case then I suspect there's a logic error in the new version of the ergm.pl function somewhere.

smjenness commented 5 years ago

Just curious, but as a temporary workaround, can you pass in a non-empty (some edges manually added) network?

dth2 commented 5 years ago

Yes this does appear to be an acceptable workaround for the moment.

From: Samuel Jenness [mailto:notifications@github.com] Sent: Tuesday, November 20, 2018 4:34 PM To: statnet/ergm ergm@noreply.github.com Cc: Deven T. Hamilton dth2@uw.edu; Mention mention@noreply.github.com Subject: Re: [statnet/ergm] Error in estimation with ERGM 3.9.4 and tergm 3.5.2 (#36)

Just curious, but as a temporary workaround, can you pass in a non-empty (some edges manually added) network?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/statnet/ergm/issues/36#issuecomment-440481214, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJ-_MSkPFgJPzq7N9jHoXj2Bp_oKZS2hks5uxJ9hgaJpZM4Ycwwk.

krivit commented 5 years ago

I've finally had a chance to run this, and I am not reproducing the error with the latest ergm and tergm master.

Regarding the change in ergm.pl(), I am not quite sure what to do with it. It was never meant to be an exported function, so not much thought had been put into coming up with good arguments and documenting it.

dth2 commented 5 years ago

I am confirming the Error with the master versions of ergm and tergm. Included below are 1) my session Info confirming the versions of ergm and tergm, 2) the code I used to generate the error, 3) the error and 4) the place in ergm where the error occures.

1) Session Info

R version 3.5.1 (2018-07-02) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default BLAS: /usr/lib/openblas-base/libblas.so.3 LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] parallel stats graphics grDevices utils datasets methods base

other attached packages: [1] EpiModelHIV_1.5.0 tergmLite_1.2.0 EpiModelHPC_1.5.0 EpiModel_1.6.5 tergm_3.6.0-1659 ergm_3.10.0-4700 devtools_1.13.6
[8] networkDynamic_0.9.0 network_1.14-355 deSolve_1.21 MASS_7.3-50

loaded via a namespace (and not attached): [1] tidyselect_0.2.5 bindata_0.9-19 lpSolve_5.6.13 purrr_0.2.5 lattice_0.20-35 colorspace_1.3-2 yaml_2.1.19
[8] rlang_0.2.2 e1071_1.6-8 pillar_1.2.3 glue_1.3.0 withr_2.1.2 RColorBrewer_1.1-2 bindrcpp_0.2.2
[15] trust_0.1-7 foreach_1.4.4 bindr_0.1.1 plyr_1.8.4 robustbase_0.93-3 munsell_0.5.0 gtable_0.2.0
[22] mvtnorm_1.0-8 codetools_0.2-15 coda_0.19-2 memoise_1.1.0 doParallel_1.0.11 class_7.3-14 DEoptimR_1.0-8
[29] Rcpp_0.12.19 scales_0.5.0 ggplot2_2.2.1 digest_0.6.15 dplyr_0.7.7 grid_3.5.1 tools_3.5.1
[36] magrittr_1.5 lazyeval_0.2.1 tibble_1.4.2 ape_5.1 pkgconfig_2.0.2 Matrix_1.2-14 assertthat_0.2.0
[43] iterators_1.0.9 statnet.common_4.1.4 R6_2.3.0 nlme_3.1-137 compiler_3.5.1

2) Simulation Script

library(devtools) devtools::install_github("statnet/network") devtools::install_github("statnet/ergm") devtools::install_github("statnet/tergm") devtools::install_github("statnet/tergmLite/") devtools::install_github("statnet/EpiModelHIV",ref="campcl") library(EpiModelHIV)

Time unit for simulation, relative to 1 day

time.unit <- 7

Population size by race

num.B <- 13500 num.W <- 13500 num.B.msm <- 10500 num.W.msm <- 10500 num.B.asmm <- 3000 num.W.asmm <- 3000

deg.mp.B <- deg.mp.W <- (matrix(c(0.506, 0.151, 0.053, 0.207, 0.061, 0.022), byrow = TRUE, nrow = 2) + matrix(c(0.435, 0.184, 0.095, 0.233, 0.033, 0.020), byrow = TRUE, nrow = 2))/2

Mean degree for ASMM Calculated conditional on age of enrty and debut.

deg.asmm <- 0.162 deg.mp.B[1,1]<-deg.mp.B[1,1]+0.01123101 deg.mp.B[1,2]<-deg.mp.B[1,2]-0.01123101 deg.mp.W <- deg.mp.B

Mean degree for ASMM Calculated conditional on age of enrty and debut.

deg.asmm <- 0.162

The fraction of MSM ties moving to the ASMM network.

cross.frac<-0.01123101

Adult Instant rates

mdeg.inst.B <- mdeg.inst.W <- (matrix(c(0.010402, 0.012954, 0.011485, 0.007912, 0.007424, 0.007424), byrow = TRUE, nrow = 2) + matrix(c(0.008186, 0.012017, 0.013024, 0.008151, 0.008341, 0.008341), byrow = TRUE, nrow = 2))/2

ADULT Quintile distribution of overall AI rates

qnts.W <- qnts.B <- c(0, 0.00100413270621469, 0.00536670889830508, 0.0101956474689266, 0.0315210354777778)

Proportion in same-race partnerships (main, casl, inst)

prop.hom.mpi.B <- prop.hom.mpi.W <- (c(0.9484, 0.9019, 0.9085) + c(0.9154, 0.8509, 0.8944))/2

ADULT Mean age diffs (main, casl, inst)

sqrt.adiff.BB <- c(0.417, 0.498, 0.456) sqrt.adiff.BW <- c(0.454, 0.629, 0.585) sqrt.adiff.WW <- c(0.520, 0.632, 0.590)

ADULT Mean durations

rates.main <- mean(c(0.00287612937991679, 0.00269183371091241, 0.00180272348650181)) rates.pers <- mean(c(0.00761700198417522, 0.00350074333616134, 0.00693147180559945))

durs.main <- 1/rates.main durs.pers <- 1/rates.pers

ASMM Mean age diffs (place holder until we have data)

sqrt.adiff.asmm <- .2

ASMM durations.

rates.asmm <- .049505 durs.asmm <- 1/rates.asmm

Minimum age at entry .

birth.age<-13 out.age.prob<-c(0.44, 0.13, 0.12, 0.11, 0.10, 0.10) debut.entry.prob<-.5062467131 debut.prob<- 1-(1-.0004864819)^7

Quintile distribution weights for risk groups for partnership formation

riskg.asmm <- c(.039682539, 0.079365079, 0.119047619 , 0.182539682, 0.579365079)

ages<-13:39 ages.asmm<-13:18 ages.yamsm<-19:25 ages.oamsm<-26:39 ages.msm<-19:39

Age-sex-specific mortality rates

asmr.B <- c(rep(0, 12), 1-(1-c(rep(0.00159, 6), rep(0.00159, 7), rep(0.00225, 10), rep(0.00348, 4)))^(1/(365/time.unit)), 1) asmr.W <- c(rep(0, 12), 1-(1-c(rep(0.00103, 6), rep(0.00103, 7), rep(0.00133, 10), rep(0.00214, 4)))^(1/(365/time.unit)), 1)

I, R, V role frequencies

role.B.prob.asmm <- c(0.145, 0.282, 0.573) role.W.prob.asmm <- c(0.145, 0.282, 0.573)

I, R, V role frequencies

role.B.prob.msm <- role.W.prob.msm <- (c(0.242, 0.321, 0.437) + c(0.228, 0.228, 0.544))/2 r.to.i <- (0.282 - ((0.321 + 0.228) /2 )) / 0.282 v.to.i <- (0.573 - ((0.437 + 0.544) /2 )) / 0.573 role.shift <- c(r.to.i, v.to.i)

Create meanstats

st <- calc_nwstats_msm( method = 1, time.unit = time.unit, num.B = num.B, num.W = num.W, num.B.msm = num.B.msm, num.W.msm = num.W.msm, num.B.asmm = num.B.asmm, num.W.asmm = num.W.asmm, deg.mp.B = deg.mp.B, deg.mp.W = deg.mp.W, mdeg.inst.B = mdeg.inst.B, mdeg.inst.W = mdeg.inst.W, deg.asmm = deg.asmm, cross.frac = cross.frac, qnts.B = qnts.B, qnts.W = qnts.W, prop.hom.mpi.B = prop.hom.mpi.B, prop.hom.mpi.W = prop.hom.mpi.W, balance = "mean", sqrt.adiff.BB = sqrt.adiff.BB, sqrt.adiff.WW = sqrt.adiff.WW, sqrt.adiff.BW = sqrt.adiff.BW, sqrt.adiff.asmm = sqrt.adiff.asmm, diss.main = ~offset(edges), diss.pers = ~offset(edges), diss.asmm = ~offset(edges), durs.main = durs.main, durs.pers = durs.pers, rates.asmm = rates.asmm, durs.asmm = durs.asmm, ages = ages, ages.asmm = ages.asmm, ages.yamsm = ages.yamsm, ages.oamsm = ages.oamsm, ages.msm = ages.msm, birth.age = birth.age, out.age.prob = out.age.prob, debut.entry.prob = debut.entry.prob, debut.prob = debut.prob, asmr.B = asmr.B, asmr.W = asmr.W, role.B.prob.msm = role.B.prob.msm, role.W.prob.msm = role.W.prob.msm, role.B.prob.asmm = role.B.prob.asmm, role.W.prob.asmm = role.W.prob.asmm, role.shift = role.shift)

Initialize network

nw.asmm <- base_nw_msm(st)

Formulas

formation.asmm <- ~edges + nodefactor("riskg",base = 3)+ nodefactor("yamsm", base=1) + absdiff("sqrt.age") + offset(nodefactor("oamsm",base = 1)) + offset(nodematch("yamsm", diff = TRUE, keep = 2)) + offset(nodematch("role.class", diff = TRUE, keep = 1:2)) + offset(nodefactor("debuted",base=2))

Fit model

fit.asmm <- netest(nw.asmm, formation = formation.asmm, coef.form = c(-Inf, -Inf,-Inf, -Inf, -Inf), target.stats = st$stats.asmm, coef.diss = st$coef.diss.asmm, set.control.ergm = control.ergm(MCMC.burnin = 1e5, MCMC.samplesize = 5000, MPLE.max.dyad.types = 1e5, MCMLE.maxit = 250))

Fit model

fit.asmm <- netest(nw.asmm, formation = formation.asmm, coef.form = c(-Inf, -Inf,-Inf, -Inf, -Inf), target.stats = st$stats.asmm, coef.diss = st$coef.diss.asmm, set.control.ergm = control.ergm(MCMC.burnin = 1e5, MCMC.samplesize = 5000, MPLE.max.dyad.types = 1e5, MCMLE.maxit = 250))

3) The Error

(note that the error is different the the error related to the edgelist error when the network is empty)

Starting Monte Carlo maximum likelihood estimation (MCMLE): Iteration 1 of at most 250: Error in forkTimeout({ : could not find function "forkTimeout"** In addition: Warning message: In ergm.pl(nw = nw, fd = fd, m = m, theta.offset = init, maxMPLEsamplesize = maxMPLEsamplesize, : Too many unique dyads. MPLE is approximate, and MPLE standard errors are suspect.

4) The code in ergm where the error is generated

mainfit <- switch(control$main.method, Robbins-Monro = ergm.robmon(init, nw, model, proposal = proposal, verbose = verbose, control = control), Stochastic-Approximation = ergm.stocapprox(init, nw, model, control = control, proposal = proposal, verbose), Stepping = ergm.stepping(init, nw, model, initialfit, constraints, control = control, proposal = proposal, proposal.obs = proposal.obs, verbose = verbose, ...), MCMLE = ergm.MCMLE(init, nw, model, initialfit = (initialfit <- NULL), control = control, proposal = proposal, proposal.obs = proposal.obs, verbose = verbose, response = response, ...), stop("Method ", control$main.method, " is not implemented."))

krivit commented 5 years ago

Thanks, @dth2 . Can you try updating statnet.common to master as well? forkTimeout() is a new function in there.

sgoodreau commented 5 years ago

Thanks @krivit. To be clear - we aren't calling ergm.pl directly; the EpiModel function netest that you see in the code calls ergm and sometimes stergm (if edapprox=FALSE). It's just that I traced the error that is appearing down to ergm.pl.

"Regarding the change in ergm.pl(), I am not quite sure what to do with it. It was never meant to be an exported function, so not much thought had been put into coming up with good arguments and documenting it."

sgoodreau commented 5 years ago

@krivit and #dth6, I am confirming that the problem appears on my machine when I have the CRAN versions of ergm, tergm, and statnet.common. I will try master now.

sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] EpiModelHIV_1.5.0 EpiModelHPC_1.5.0 EpiModel_1.2.9 tergmLite_1.0.0
[5] tergm_3.5.2 ergm_3.9.4 networkDynamic_0.9.0 network_1.13.0.1
[9] deSolve_1.21

loaded via a namespace (and not attached): [1] Rcpp_1.0.0 pillar_1.3.0 compiler_3.5.1 DEoptimR_1.0-8
[5] RColorBrewer_1.1-2 bindr_0.1.1 class_7.3-14 iterators_1.0.10
[9] tools_3.5.1 tibble_1.4.2 nlme_3.1-137 lattice_0.20-35
[13] pkgconfig_2.0.2 rlang_0.3.0.1 foreach_1.4.4 Matrix_1.2-14
[17] rstudioapi_0.8 yaml_2.2.0 parallel_3.5.1 mvtnorm_1.0-8
[21] bindrcpp_0.2.2 e1071_1.7-0 bindata_0.9-19 coda_0.19-2
[25] dplyr_0.7.8 trust_0.1-7 grid_3.5.1 tidyselect_0.2.5
[29] robustbase_0.93-3 glue_1.3.0 R6_2.3.0 purrr_0.2.5
[33] magrittr_1.5 codetools_0.2-15 MASS_7.3-50 assertthat_0.2.0
[37] lpSolve_5.6.13 ape_5.2 lazyeval_0.2.1 doParallel_1.0.14
[41] statnet.common_4.1.4 crayon_1.3.4

dth2 commented 5 years ago

I am trying with master now as well. Including master statnet.common.

From: Steven Goodreau [mailto:notifications@github.com] Sent: Monday, November 26, 2018 10:21 AM To: statnet/ergm ergm@noreply.github.com Cc: Deven T. Hamilton dth2@uw.edu; Mention mention@noreply.github.com Subject: Re: [statnet/ergm] Error in estimation with ERGM 3.9.4 and tergm 3.5.2 (#36)

@krivithttps://github.com/krivit and #dth6, I am confirming that the problem appears on my machine when I have the CRAN versions of ergm, tergm, and statnet.common. I will try master now.

sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C [5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] EpiModelHIV_1.5.0 EpiModelHPC_1.5.0 EpiModel_1.2.9 tergmLite_1.0.0 [5] tergm_3.5.2 ergm_3.9.4 networkDynamic_0.9.0 network_1.13.0.1 [9] deSolve_1.21

loaded via a namespace (and not attached): [1] Rcpp_1.0.0 pillar_1.3.0 compiler_3.5.1 DEoptimR_1.0-8 [5] RColorBrewer_1.1-2 bindr_0.1.1 class_7.3-14 iterators_1.0.10 [9] tools_3.5.1 tibble_1.4.2 nlme_3.1-137 lattice_0.20-35 [13] pkgconfig_2.0.2 rlang_0.3.0.1 foreach_1.4.4 Matrix_1.2-14 [17] rstudioapi_0.8 yaml_2.2.0 parallel_3.5.1 mvtnorm_1.0-8 [21] bindrcpp_0.2.2 e1071_1.7-0 bindata_0.9-19 coda_0.19-2 [25] dplyr_0.7.8 trust_0.1-7 grid_3.5.1 tidyselect_0.2.5 [29] robustbase_0.93-3 glue_1.3.0 R6_2.3.0 purrr_0.2.5 [33] magrittr_1.5 codetools_0.2-15 MASS_7.3-50 assertthat_0.2.0 [37] lpSolve_5.6.13 ape_5.2 lazyeval_0.2.1 doParallel_1.0.14 [41] statnet.common_4.1.4 crayon_1.3.4

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/statnet/ergm/issues/36#issuecomment-441743347, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJ-_MaKVs5ZGx1XRsPMnFCbCggxwTLbTks5uzDD-gaJpZM4Ycwwk.

sgoodreau commented 5 years ago

With master versions of ergm', 'stergm', 'network',and 'statnet.common, I get the following error:

Error in .C("SAN_wrapper", as.integer(nedges), as.integer(tails), as.integer(heads), : Incorrect number of arguments (29), expecting 32 for 'SAN_wrapper'

Based upon the diagnosis I did above, and the success of Sam's proposed workaround, it really does seem that the latest edits to ergm.pl cause it to not create list elements named heads and tails when the network is empty, but then the same function later down expects there to be heads and tails elements regardless, and breaks when there isn't.

dth2 commented 5 years ago

With the following session info it works for me.

sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default BLAS: /usr/lib/openblas-base/libblas.so.3 LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] EpiModelHIV_1.5.0 tergmLite_1.2.0 EpiModelHPC_1.5.0 EpiModel_1.7.0 tergm_3.6.0-1659 [6] ergm_3.10.0-4700 networkDynamic_0.9.0 network_1.14-355 deSolve_1.21 devtools_1.13.6 [11] MASS_7.3-50

loaded via a namespace (and not attached): [1] Rcpp_0.12.19 ape_5.2 mvtnorm_1.0-8 lattice_0.20-35 [5] class_7.3-14 assertthat_0.2.0 rprojroot_1.3-2 digest_0.6.18 [9] foreach_1.4.4 R6_2.3.0 plyr_1.8.4 backports_1.1.2 [13] evaluate_0.10.1 coda_0.19-2 e1071_1.6-8 httr_1.3.1 [17] ggplot2_3.1.0 pillar_1.2.3 rlang_0.3.0.1 lazyeval_0.2.1 [21] curl_3.2 Matrix_1.2-14 bindata_0.9-19 rmarkdown_1.10 [25] stringr_1.3.1 munsell_0.5.0 compiler_3.5.1 pkgconfig_2.0.2 [29] htmltools_0.3.6 tidyselect_0.2.5 tibble_1.4.2 lpSolve_5.6.13 [33] statnet.common_4.2.0-205 codetools_0.2-15 dplyr_0.7.8 withr_2.1.2 [37] grid_3.5.1 nlme_3.1-137 gtable_0.2.0 git2r_0.21.0 [41] magrittr_1.5 scales_1.0.0 stringi_1.2.3 doParallel_1.0.14 [45] bindrcpp_0.2.2 robustbase_0.93-3 trust_0.1-7 RColorBrewer_1.1-2 [49] iterators_1.0.10 tools_3.5.1 glue_1.3.0 DEoptimR_1.0-8 [53] purrr_0.2.5 parallel_3.5.1 yaml_2.1.19 colorspace_1.3-2 [57] memoise_1.1.0 knitr_1.20 bindr_0.1.1

From: Steven Goodreau [mailto:notifications@github.com] Sent: Monday, November 26, 2018 10:41 AM To: statnet/ergm ergm@noreply.github.com Cc: Deven T. Hamilton dth2@uw.edu; Mention mention@noreply.github.com Subject: Re: [statnet/ergm] Error in estimation with ERGM 3.9.4 and tergm 3.5.2 (#36)

With master versions of ergm', 'stergm', 'network',and 'statnet.common, I get the following error:

Error in .C("SAN_wrapper", as.integer(nedges), as.integer(tails), as.integer(heads), : Incorrect number of arguments (29), expecting 32 for 'SAN_wrapper'

Based upon the diagnosis I did above, and the success of Sam's proposed workaround, it really does seem that the latest edits to ergm.pl' cause it to not create list elements named headsandtailswhen the network is empty, but then the same function later down expects there to beheadsandtails` elements regardless, and breaks when there isn't.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/statnet/ergm/issues/36#issuecomment-441749862, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJ-_MepmVuI8Ha-RKz6i8mIK5lrWvXSGks5uzDWhgaJpZM4Ycwwk.

sgoodreau commented 5 years ago

sorry everyone - looks like I accidentally closed the issue. Since I'm still getting an error (even though Deven isn't), it should still be active.

sgoodreau commented 5 years ago

Ok, I did a fresh install of all of the master versions of the packages, and now it works for me.

So, my understanding of where things currently stand is that Deven's code works with the master version of packages, but not with the current CRAN version for everything. @dth2, is that true for you?

If so, then, @krivit, I'm wondering what the plan is for release of master. Since underlying Deven's code is a fairly vanilla call to ergm, this suggests that there is currently a bug out in the wild in the CRAN version that affects large networks/complex models. Or do you interpret this all differently?

dth2 commented 5 years ago

Yes, my code works with the master versions of everything but not the CRAN versions. As an aside this code also works with older versions of ergm and tergm. The issue arose because other packages I am using for a different project depend on ergm and now require the current version of ergm.

From: Steven Goodreau [mailto:notifications@github.com] Sent: Monday, November 26, 2018 1:39 PM To: statnet/ergm ergm@noreply.github.com Cc: Deven T. Hamilton dth2@uw.edu; Mention mention@noreply.github.com Subject: Re: [statnet/ergm] Error in estimation with ERGM 3.9.4 and tergm 3.5.2 (#36)

Ok, I did a fresh install of all of the master versions of the packages, and now it works for me.

So, my understanding of where things currently stand is that Deven's code works with the master version of packages, but not with the current CRAN version for everything. @dth2https://github.com/dth2, is that true for you?

If so, then, @krivithttps://github.com/krivit, I'm wondering what the plan is for release of master. Since underlying Deven's code is a fairly vanilla call to ergm, this suggests that there is currently a bug out in the wild in the CRAN version that affects large networks/complex models. Or do you interpret this all differently?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/statnet/ergm/issues/36#issuecomment-441808735, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJ-_MQku_uBGLJR7YMcnPwZ3riwLIcHbks5uzF9kgaJpZM4Ycwwk.

krivit commented 5 years ago

I can probably start pushing updates in the week after next. Would that work?

martinamorris commented 5 years ago

works for me.

Carter and I are teaching 3 statnet workshops at NASN tomorrow...

sgoodreau commented 5 years ago

Works for me, thanks!

krivit commented 5 years ago

OK. Does this mean that this is fixed in master?

sgoodreau commented 5 years ago

This issue appears to be fixed in master, at least as far as Deven's and my testing are concerned.

That said, the fix that has me most concerned, though, and which isn't currently in master, is the solution mentioned in #37 . And the reason it has me most concerned isn't because of the issue in #37 itself (which is a problem, to be sure) but rather because it also solves the very longstanding issue that causes major problems that is posted as issue 1 (Debugging strange netdx results) in the EpiModel/EpiModel-MIDAS repository. We've pinged you (@krivit) over there a few times, but you haven't weighed in on the various questions - if it's because you don't have access to that organization, please let us know.

Fixing these issues (#37 in this repository, number 1 in EpiModel-MIDAS, with what appears to be just a single fix) is crucial before the upcoming release.

Much thanks.

krivit commented 5 years ago

@sgoodreau, hanks for the explanation. I've been away at a conference for the past week, so I've only been skimming the issue.