search

statnet / ergm

Fit, Simulate and Diagnose Exponential-Family Models for Networks
Other
98 stars 37 forks source link

Consider calling glm.fit() instead of glm() #569

Open mbojan opened 4 months ago

mbojan commented 4 months ago

In

https://github.com/statnet/ergm/blob/1f4401ed73356cbf89c2f35fd35d6a981f4caea0/R/ergm.mple.R#L100-L101

consider calling glm.fit() directly rather than glm(). Experiments with biggish data show that it might cut the computing time by half.

krivit commented 3 months ago

@mbojan , can you test to see if this works?

krivit commented 2 months ago

@mbojan , I'll be submitting an update to ergm in the next few days. If you want this to go in, let me know ASAP.

mbojan commented 2 months ago

Oh dear, I have a week of workshops, including ergms. Can we release next week? The principle answer is yes, but haven't tested yet.

krivit commented 1 month ago

Oh dear, I have a week of workshops, including ergms. Can we release next week? The principle answer is yes, but haven't tested yet.

OK, can you get it done in the next day or two?

krivit commented 1 month ago

@mbojan ?

krivit commented 1 month ago

@AdrienLeGuillou , you often fit MPLE to large networks, right? Can you by any chance test this?

AdrienLeGuillou commented 1 month ago

I just ran a quick test on a smaller 10k nodes network using this branch. It worked fine. I can't tell if it was faster or not as I usually work with "Stochastic-Approximation" on these smaller local tests. I can try on the HPC with our 3 - 100k nodes networks and compare the time it takes.

AdrienLeGuillou commented 1 month ago

I just realized that ergm.mple is called whatever the main.method we use. Therefore I can confirm that it works perfectly on our 10k nodes networks. It takes a very similar amount of time to fit the networks with both version as the MPLE step is not the longest part anyways.

AdrienLeGuillou commented 1 month ago

I confirm it also works on the 100k nodes network. It was actually longer with glm.fit. But the difference was on the number of MCMLE iterations.

mbojan commented 1 month ago

Thanks @AdrienLeGuillou . @krivit don't merge, leave as is. I need to dig out the script where I think I noticed the difference.