search

danielegrattarola / keras-gat

Keras implementation of the graph attention networks (GAT) by Veličković et al. (2017; https://arxiv.org/abs/1710.10903)
MIT License
475 stars 123 forks source link

Why I got loss value nan in working with citeseer data #28

Closed gaudelbijay closed 1 year ago

gaudelbijay commented 4 years ago

Epoch 1/100 3327/3327 - 2s - loss: nan - categorical_crossentropy: nan - acc: 0.1667 - val_loss: nan - val_categorical_crossentropy: nan - val_acc: 0.0580 Epoch 2/100 3327/3327 - 2s - loss: nan - categorical_crossentropy: nan - acc: 0.1667 - val_loss: nan - val_categorical_crossentropy: nan - val_acc: 0.0580

danielegrattarola commented 4 years ago

Hi,

can you provide more details on this? Did you only change the dataset? What versions of TF and Keras are you running?

Cheers. Daniele

gaudelbijay commented 4 years ago

I am running tensorflow 2.2.0 , works fine for cora dataset and reproduce the result, but didn't work for citeseer and pubmed dataset.

danielegrattarola commented 4 years ago

I can run the code on TF 2.2 and Keras 2.4.3 and it trains as expected with no NaNs. Are you running on GPU with some particular setup (e.g. restricted precision)? Does it work if you train on CPU only?

FYI, expect the code not to work on Pubmed because this implementation of GAT is very inefficient. You can have a look at Spektral, my library for GNNs in tf.keras. We have an efficient and up-to-date implementation of GAT (see docs) that will work on Pubmed as well.

gaudelbijay commented 4 years ago

Why this gives very bad result in the inductive settings for the citeseer dataset. Is there any specific reason?

danielegrattarola commented 4 years ago

Can you be a bit more specific? Citeseer is a transductive task and the code should reproduce the results from the paper as-is, I'm not sure I understand that problem that you are encountering.

Anyway, were you able to solve the original issue?

Cheers

gaudelbijay commented 4 years ago

Can't we trained Citeseer in inductive settings? Like Citation data in GraphSAGE paper?

danielegrattarola commented 4 years ago

The code should work both for transductive and inductive learning, I'm not sure why you are getting unexpected results. Could it be something with your data pipeline? What results are you trying to replicate?

Cheers