Review Graph Neural Network and Deploy with More Features

[ijyliu]

Review the resources below, then the code in https://github.com/current12/Stat-222-Project/tree/main/Code/Modelling/Graph%20Neural%20Network. Let me know what you think of some design decisions:

• Transductive vs. Inductive - currently inductive, should we switch to transductive? I don't really want to do both. In the original paper the performance is often similar but in some cases transductive seems to pull ahead.
• We're using the (small) network of company mentions because I thought it was more intuitive than the Doc2Vec approach in the original paper https://pubsonline.informs.org/doi/epdf/10.1287/ijds.2022.00018. But yeah, it's really small so we only really get predictions for a small number of company by fixed quarter dates, kind of limiting the comparability to our other results (we can really only say anything about the firms that mention someone else in a given call).
• Include or exclude previous rating? Currently set to exclude. Or should we model changes instead?
• We can try to extract node embeddings and feed them to some other classifier instead of doing end-to-end, but I don't really want to do this because it would involve grid search and stuff.
• In section 4.2 of the Graph NN paper, they discuss some node features based on the graph that are included: degree centrality, eigenvector, and clustering coefficient. Should we incorporate these? It'd require some research or more inspection of the code to figure out how. I don't think they clearly improve performance in the original paper.

Copy + write code for the other feature sets - 2, 3, etc.

Resources

GNN Resources

• Intro to GNNs 1: https://www.youtube.com/watch?v=fOctJB4kVlM&t=8s
• Intro to GNNs 2: https://www.youtube.com/watch?v=ABCGCf8cJOE
• Intro to GraphSAGE: https://www.youtube.com/watch?v=3AzphNf5ja8.

Implementation Resources

Original Paper

https://pubsonline.informs.org/doi/epdf/10.1287/ijds.2022.00018 (it's straightforward but there is a lot of stuff that isn't relevant to the GCN approach they + we implement)

• Section 4.5 - we're currently using the inductive approach, but the transductive approach could also be valid. you don't use the test data labels in either. it is just a matter of whether you want to use the test data features and graph in training. both approaches can handle new, unseen test data you add on - for transductive you retrain the network with the new data connections and features, inductive you add connections but use existing parameters and ignore new features.
• Details on node aggregation and update:

Code

• Documentation for the model: https://docs.dgl.ai/en/1.1.x/generated/dgl.nn.pytorch.conv.SAGEConv.html
• We're currently not using weights (though we could, with the number of mentions being the weight).
• For each layer: in_feats & out_feats dimensions, aggregator (see above specification - implemented with argument 'pool'), no dropout, yes bias, no normalization, no activation on updated node features
• In terms of broader structure, two layers is specified as n_layers, and as a result we have an input layer, a hidden layer, and an output layer. input layer width starts at number of features (starting with one). output layer gets us down to scores for each class, and then we do cross entropy loss to classify.

• Currently takes only a minute to run locally with only the altman z feature, obviously we should scale up. Not clear if GPU will be necessary since the graph is pretty small, but there's a script for an A100 on SCF if we need it.

[ijyliu]

@seanzhou1207 @current12 @OwenLin2001

check this out if you have time and are interested, otherwise I will make the decisions and finish the implementation when I finish the rest of my writeup

[ijyliu]

implemented all of this. ended up making the hidden layers wider and adding more dropout and weight decay, pretty much

performance seems to be between logistic regression and xgboost, pretty close to logistic regression