[Question] Proportion of training triples used

[MisaOgura]

Hello!

First of all thanks so much for this awesome publication & codebase.

I'm in the process of tweaking a config for training NBFNet, and trying to understand the proportion of the training triples used when training on ogbl-biokg using the provided config config/knowledge_graph/ogbl-biokg.yaml.

Since batch_size: 8, batch_per_epoch: 200 and num_epoch: 10, and the number of training triples in ogbl-biokg being 4,762,678, is it correct to assume that only (8 * 200 * 10)/4,762,678 = 0.000335... ≈ 0.34% of the training triples is used for the entire training run?

It seems very small and I'm most likely missing some vital implementation details - I'd appreciate your help.

Thanks so much!

[KiddoZhu]

Yes. If you use 1 GPU, that's right. If you use 4 GPUs, then simply multiply that by 4.

We found that for large datasets, only a small number of training samples is enough to get the model converge to a good performance. This is possibly because NBFNet only learns the relations and doesn't learn anything about the entities. As long as the number of training samples is enough w.r.t. the number of relations in the dataset, it is fine.