JuliaGast / TGB2

Temporal Graph Benchmark project repo
2 stars 0 forks source link

Integrate Recurrency Baseline #3

Closed JuliaGast closed 5 months ago

JuliaGast commented 6 months ago
JuliaGast commented 6 months ago
shenyangHuang commented 6 months ago

let's discuss tomorrow but starting with MRR ranking over all possible destination could be a good starting point to get an idea of what kind of performance we have on the dataset. We don't want datasets where methods already has overly high performance to begin with

JuliaGast commented 5 months ago

todos:

JuliaGast commented 5 months ago

YAGO: without training:

   {
        "model": "RecurrencyBaseline",
        "train_flag": false,
        "data": "tkgl-yago",
        "run": 1,
        "seed": 1,
        "mrr": 0.8588335514068604,
        "hits10": 0.9259546399116516,
        "test_time": 175.950186,
        "tot_train_val_time": 190.072089
    }

with training:

    {
        "model": "RecurrencyBaseline",
        "train_flag": "True",
        "data": "tkgl-yago",
        "run": 1,
        "seed": 1,
        "mrr": 0.9091410040855408,
        "hits10": 0.9302738904953003,
        "test_time": 104.039734,
        "tot_train_val_time": 922.78976
    }

POLECAT mini:

    {
        "model": "RecurrencyBaseline",
        "train_flag": false,
        "data": "tkgl-polecat",
        "run": 1,
        "seed": 1,
        "mrr": 0.14627420902252197,
        "hits10": 0.21407249569892883,
        "test_time": 38.191202,
        "tot_train_val_time": 44.85462
    },
    {
        "model": "RecurrencyBaseline",
        "train_flag": "True",
        "data": "tkgl-polecat",
        "run": 1,
        "seed": 1,
        "mrr": 0.1447719931602478,
        "hits10": 0.21252413094043732,
        "test_time": 19.861222,
        "tot_train_val_time": 210.635383
    }
]