Selecting accurate enough models

[kristinmg]

How can one show that the model selected for evaluating a network is modeling it's characteristics well enough for the results to be significant?

[dlee138]

I would say it depends on your expectations and what you want to do with your model. There isn't a clear cut value that tells you that a model is correct, but some models are better than others, depending on what you want to do. You can use a "poor" model (say the Bernoulli model mentioned in class), but that model is fine if your goal is to distinguish adults and infants by comparing their total number of connections. However, the same model would probably not work for evaluating other characteristics. It is ultimately up to the researcher to decide what threshold they want for statistical significance and a model may pass this threshold for some characteristics, but not others.

[akim1]

I think this is the way people generally do this.

Let's say you have 50 participants in your clinical study that tries to predict the onset of a disease based on measurements that can be easily made in a clinic. In 25 of those patients, you measure certain quantities (e.g., level of a certain protein) and try to create a relationship between that quantity and the relevant phenotype (e.g., onset of a disease). The relationship is given by a model with a bunch of unknown parameters (e.g., 'p' in the Bernoulli distribution). You use these 25 patients to determine what the parameters actually are.

And then you take your model with the computed parameters and try to predict whether this disease will arise in the remaining 25 of your patients. If you can predict with some reasonable certainty, then you have a "good enough" model. If not, then your model is probably not that great.