search

JuliaGast / TGB2

Temporal Graph Benchmark project repo
1 stars 0 forks source link

needed: 1-vs-all time-aware filtered for thgl-github #35

Closed JuliaGast closed 2 months ago

JuliaGast commented 3 months ago

I think, for thgl-github, the negative sampling is not great Reason: the recurrency baseline should not be good for this dataset. It has a rather low recurrency degree, and applying the relaxed recurrency baseline only seems to lead to not good results either Still, it gets a rather ok mrr of 0.17 In the examples that I checked, for a node-relation combi, there have been 2000-6000 different objects -> 1000 is a small subsample of the possible nodes Given that this is our smallest dataset, I strongly suggest that we do not do any negative sampling here. We have a very small recurrency degree. I expect this dataset will be very hard. hm.

JuliaGast commented 3 months ago

fixed with commit https://github.com/JuliaGast/TGB2/commit/ef725b3fb38e0a93e335c3b8506ef23f842efaa4

@shenyangHuang could you please check and integrate in main branch?