search

snap-stanford / ogb

Benchmark datasets, data loaders, and evaluators for graph machine learning
https://ogb.stanford.edu
MIT License
1.89k stars 397 forks source link

[Request] Please remove the warning for older versions of OGB #428

Closed diningphil closed 1 year ago

diningphil commented 1 year ago

Dear OGB team,

Would it be possible to remove the following warning

WARNING The OGB package is out of date. Your version is 1.3.5, while the latest version is 1.3.6

from future OGB versions? It affects CLI/GUI based programs that import your package. Or, is there a way to disable this specific warning?

Many thanks!

weihua916 commented 1 year ago

Hi thanks for reaching out. Hmm why don't you update your version? As the dataset package, it is important for us to make sure everyone is using the up-to-date package.

diningphil commented 1 year ago

Thanks for the prompt answer. Do you mean that you want to be sure everyone can access the latest benchmarks you add?

If that is the only reason, and new releases do not impact the reproducibility of previously added benchmarks, then imho it is a bit of an overkill to add such a message. The user could just update the package and get access to the new features without being reminded every time.

I think the same goes when some main change is introduced: inviting (or urging) users to update the package with that warning could result in irreproducible experiments if subtle changes to the evaluation code were made, and so it would be better to safely upgrade in a second moment, having the user evaluate the changes you introduced.

Maybe you could instead use a changelog.md file so that users can readily see if something of their interest changed? This is clearly not a critical issue, but depending on the rate of updates it can become annoying for the user, especially those using CLIs. I have never seen such update warnings being used in other pip packages, and it is always good to stay up to date, but I would not force the users' hand by reminding them every time. This is just my perspective, but I hope you will take this into consideration for future releases.

I am closing the issue since I will leave the final decision to you. I can always disable the warning directly from the package source. But again, I think it is a bit too much for the user.