add gatv2 model - Githubissues

awslabs / dgl-lifesci

Python package for graph neural networks in chemistry and biology

Apache License 2.0

696 stars 144 forks source link

add gatv2 model #202

Closed rgasper closed 1 year ago

rgasper commented 1 year ago

Description of changes:

Added GATv2 Layers and GATv2 predictor from DGL, described in HOW ATTENTIVE ARE GRAPH ATTENTION NETWORKS?. Added an implementation test copied from test_gat
- implemented output of attention values for model introspection as used in the paper
- Not sure if outputting these as a list is the right call, if someone more experienced in pytorch fundamentals/api designs has a suggestion I'm open to it
fixed an import bug in test_new_dataset.py that was preventing pytest discovery when run independent of the task_unit_test.sh script

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

mufeili commented 1 year ago

Thank you for your contribution! Have you tried running your model using an example script like this and observed a reasonable accuracy number?

rgasper commented 1 year ago

Thank you for your contribution! Have you tried running your model using an example script like this and observed a reasonable accuracy number?

I've not - I can give that a try soon

rgasper commented 1 year ago

@mufeili - just copying the optimized hyperparameters from GAT_canonical.json & GAT_attentivefp.json for Tox21, I got:

python classification.py -d Tox21 -mo GATv2 -f canonical

...

EarlyStopping counter: 3 out of 3
epoch 9/1000, validation roc_auc_score 0.6908, best validation roc_auc_score 0.6924
val roc_auc_score 0.6924
test roc_auc_score 0.6715

And attentive_fp:

python classification.py -d Tox21 -mo GATv2 -f attentivefp

...

EarlyStopping counter: 3 out of 3
epoch 8/1000, validation roc_auc_score 0.6928, best validation roc_auc_score 0.7139
val roc_auc_score 0.7139
test roc_auc_score 0.6937

Comparing to the GAT scores, this seems reasonable for an un-optimized attempt

rgasper commented 1 year ago

If you're intending to approve, LMK and I'll go script a bit to run the optimization and save GATv2 hyperparameters for all the dataset/feature combos

mufeili commented 1 year ago

Thanks for the great efforts. Let's first merge the model architectures without these configuration files. Is it ok for you if I make a few modifications directly on your branch? You may want to have another copy of your current branch first.

rgasper commented 1 year ago

Go ahead!