What are the main differences between Graphormer v1.0 and v2.0?

microsoft / Graphormer

Graphormer is a general-purpose deep learning backbone for molecular modeling.

MIT License

2.08k stars 334 forks source link

What are the main differences between Graphormer v1.0 and v2.0? #89

Closed seonwoo-min closed 2 years ago

seonwoo-min commented 2 years ago

Hi, I have a quick question about Graphormer v2.0. What are the main differences between Graphormer v1.0 and v2.0?

Are there any model architecture or training hyperparameter updates?
Which factors attribute to the performance differences in PCQM4Mv1? (https://github.com/microsoft/Graphormer/tree/main/examples/property_prediction#pcqm4mv1)
Do you have specific reasons to use fairseq backbone? Is it for the mixed-precision training?

Thanks in advance!

shiyu1994 commented 2 years ago

@mswzeus

Are there any model architecture or training hyperparameter updates?

Yes. There are some differences in implementation between v1.0 and v2.0.

Do you have specific reasons to use fairseq backbone? Is it for the mixed-precision training?

We switch to fairseq backbone for efficiency. And it is easier to schedule a distributed training task with fairseq. And it is easy to extend with new models and tasks.