search

desmarais-lab / Supreme_Court_Citation_Network

ERGM for the SCOTUS Citation Network
4 stars 1 forks source link

use splines instead of polynomials for interactions #6

Open bdesmarais opened 6 years ago

bdesmarais commented 6 years ago

@schmid86, we can proceed with polynomials only---no splines, but we should do some predictive experiments to see if model fit improves with the use of higher-order polynomials. Would you please run the cross-validation experiments to compare models with 0-4 order polynomials? Ideally, we want to run the fit experiments until we find an order of polynomials that does not fit better than the previous order (i.e., cubic splines fitting better than models with time^4 interactions). Let me know if you have questions about this.

schmid86 commented 6 years ago

@bdesmarais Sounds good. I will start the simulations today, but they will probably not be finished by our meeting time tomorrow

bdesmarais commented 6 years ago

How has this gone? If I recall correctly, I wrote the out-of-sample test script for SPSA. Let me know if you want me to work on or review the code for this experiment.

schmid86 commented 6 years ago

Yes, I have the code and I am able to follow it, so I should be able to use/apply your code. I was not able to run the out-of-sample script yet, since I don't have the inference results for the model with the new data at this point. I will run these simulations as soon as I have the new results.

bdesmarais commented 6 years ago

These predictive results are pretty surprising. It looks like none of the models improve (judging by the range of F1 scores) on the network model with linear trends. We need to double check that the linear trends improve predictive fit. Would you run this without any trends?

schmid86 commented 6 years ago

I was just working through my code and found a big mistake that was most likely the reason for the controversial results. I will rerun the holdout experiment and also include a simulation for the model without any trends.