methods contribution

[bdesmarais]

I have a proposal for an alternative methods contribution with this paper. It is not clear how to compare and select between ERGM, LSM, MMSBM, etc., since they are estimated using different inference frameworks (e.g., approximate MLE, Bayes). Some work has used out of sample prediction for comparison, but I haven't seen any simulation studies that evaluate the effectiveness of out of sample predictions for selecting among these models. As our methods contribution for this paper, what do you think of running a simulation study to see if we can effectively select the correct model using out of sample prediction? This would be in place of the study of ordinal edges.

[jpschoen]

I think this is an excellent idea. It sounds like a more generally applicable contribution. Would you you send me a couple of the reference entries for the work you mention? I'll start there to begin adding to the outline.

[bdesmarais]

That sounds great. Here are a few that use out-of-sample methods for selection/comparison with network models

http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780190228217.001.0001/oxfordhb-9780190228217-e-11

http://onlinelibrary.wiley.com/doi/10.1111/j.1541-0072.2012.00459.x/full

https://www.sciencedirect.com/science/article/pii/S0378873317300035

---

Bruce A. Desmarais Associate Professor, Department of Political Science Director, Graduate Programs in Social Data Analytics Pennsylvania State University brucedesmarais.com

On Fri, Mar 9, 2018 at 9:59 AM, John Schoeneman notifications@github.com wrote:

I think this is an excellent idea. It sounds like a more generally applicable contribution. Would you you send me a couple of the reference entries for the work you mention? I'll start there to begin adding to the outline.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpschoen/networks_chapter/issues/1#issuecomment-371836542, or mute the thread https://github.com/notifications/unsubscribe-auth/ABYXKSeLdCkJj-fPOPOKkhPBQ5eFI1hmks5tcpjlgaJpZM4SjAAl .

[jpschoen]

I changed the outline based on the new direction in the new_draft folder. While I think this new outline better suits the new simulation study, I would like your input on how we try to make sure this proposed outline doesn't cover the material that will be addressed in the preceding chapter.

[bdesmarais]

That's a good point. I am not really sure what is meant by, "Theory and Testing," but the volume editors can let us know if our outline looks too similar to this chapter.

---

Bruce A. Desmarais Associate Professor, Department of Political Science Director, Graduate Programs in Social Data Analytics Pennsylvania State University brucedesmarais.com

On Sat, Mar 10, 2018 at 9:43 PM, John Schoeneman notifications@github.com wrote:

I changed the outline based on the new direction in the new_draft folder. While I think this new outline better suits the new simulation study, I would like your input on how we try to make sure this proposed outline doesn't cover the material that will be addressed in the preceding chapter.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpschoen/networks_chapter/issues/1#issuecomment-372084728, or mute the thread https://github.com/notifications/unsubscribe-auth/ABYXKXuuJvEioy2Wl1h4hV_cJwCwV9p4ks5tdI9QgaJpZM4SjAAl .

[jpschoen]

Okay. Can you check the new outline to see if there are additional models or inference frameworks we should list? I can elaborate on the bullet points once we agree on them.

For the simulation study, I'm thinking we want to map the simulation study to the items listed in model selection criteria so as to suggest which models will better address/suit these challenges/items. Do you agree and if so, do you have suggestions for how to approach this? I'm just thinking that if we try to deal with every possible scenario in the simulations it will be too much.

I'm available everyday between 10AM-2:30PM if you want to meet on Hangouts before submitting the outline Friday.

[bdesmarais]

I'll also add pseudo likelihood to the inference frameworks. I am not sure that we will have the space to cover longitudinal models. The simulation study need not be comprehensive. One design would be to consider whether held out edge prediction was capable of identifying the correct model when the correct model was ERGM, LSM, and MMSBM---looking at binary, count, and continuous edge types. The parameter settings can be taken from published applications. Would you like to meet at 11 on Thursday to go over the one-pager? I'll take a pass at filling it out on Thursday morning.

---

Bruce A. Desmarais Associate Professor, Department of Political Science Director, Graduate Programs in Social Data Analytics Pennsylvania State University brucedesmarais.com

On Sun, Mar 11, 2018 at 10:06 PM, John Schoeneman notifications@github.com wrote:

Okay. Can you check the new outline to see if there are additional models or inference frameworks we should list? I can elaborate on the bullet points once we agree on them.

For the simulation study, I'm thinking we want to map the simulation study to the items listed in model selection criteria so as to suggest which models will better address/suit these challenges/items. Do you agree and if so, do you have suggestions for how to approach this? I'm just thinking that if we try to deal with every possible scenario in the simulations it will be too much.

I'm available everyday between 10AM-2:30PM if you want to meet on Hangouts before submitting the outline Friday.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpschoen/networks_chapter/issues/1#issuecomment-372174046, or mute the thread https://github.com/notifications/unsubscribe-auth/ABYXKVPjqVEEwKWZtqdzdykS9vnFZvOSks5tddg2gaJpZM4SjAAl .

[jpschoen]

Thursday at 11 works great.