snap-stanford / ogb

Benchmark datasets, data loaders, and evaluators for graph machine learning
https://ogb.stanford.edu
MIT License
1.89k stars 397 forks source link

another minor fix #435

Closed puririshi98 closed 1 year ago

puririshi98 commented 1 year ago

w/o fix:

File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/trainer/call.py", line 162, in _call_lightning_datamodule_hook
    return fn(*args, **kwargs)
  File "gnn.py", line 86, in setup
    self.test_idx = torch.from_numpy(dataset.get_idx_split('test-dev'))
  File "/usr/local/lib/python3.8/dist-packages/ogb/lsc/mag240m.py", line 75, in get_idx_split
    return self.__split__ if split is None else self.__split__[split]
KeyError: 'test-dev'

-> w/:

Namespace(batch_size=1024, device='0', dropout=0.5, epochs=100, evaluate=False, hidden_channels=1024, in_memory=False, model='graphsage', sizes=[25, 15])
Global seed set to 42
#Params 4884633
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Missing logger folder: logs/graphsage/lightning_logs
Reading dataset... 
Done! [622.25s]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]

  | Name      | Type       | Params
-----------------------------------------
0 | convs     | ModuleList | 3.7 M 
1 | norms     | ModuleList | 4.1 K 
2 | skips     | ModuleList | 0     
3 | mlp       | Sequential | 1.2 M 
4 | train_acc | Accuracy   | 0     
5 | val_acc   | Accuracy   | 0     
6 | test_acc  | Accuracy   | 0     
-----------------------------------------
4.9 M     Trainable params
0         Non-trainable params
4.9 M     Total params
19.539    Total estimated model params size (MB)
Sanity Checking: 0it [00:00, ?it/s]
weihua916 commented 1 year ago

This is irrelevant to pytorch lightning. Which version of OGB are you using? Does your ogb package include https://github.com/snap-stanford/ogb/blob/master/ogb/lsc/mag240m.py#L44? This should expose test-dev.

puririshi98 commented 1 year ago

oh good point, i looked at our reqs and it seems we are using an older version of ogb (updating to latest solves the issue). is it okay to add some logic to check if ogb version is older and use the fix from this PR? or do you only want the code to support latest ogb (and throw a warning /errorfor older version). I am open to either

weihua916 commented 1 year ago

I'd suggest updating to the latest one.