search

snap-stanford / ogb

Benchmark datasets, data loaders, and evaluators for graph machine learning
https://ogb.stanford.edu
MIT License
1.89k stars 397 forks source link

Evaluate rmse might be wrong? #381

Closed rayrayraykk closed 1 year ago

rayrayraykk commented 1 year ago

https://github.com/snap-stanford/ogb/blob/c8efe8ec99d11279c80f2bcdbe1567675c1c5666/ogb/graphproppred/evaluate.py#L218

rmse_list.append(np.sqrt(((y_true[is_labeled, i] - y_pred[is_labeled, i])**2).mean())) should be correct.

weihua916 commented 1 year ago

Thanks for pointing out! You are right. This is fixed at https://github.com/snap-stanford/ogb/commit/dd9e3d0ed642619528e90d312bc8b52476338c64

Btw, this should not affect any existing results, because luckily all our regression tasks are single-task (not multi-task).