Closed JuliaGast closed 5 months ago
let's discuss tomorrow but starting with MRR ranking over all possible destination could be a good starting point to get an idea of what kind of performance we have on the dataset. We don't want datasets where methods already has overly high performance to begin with
todos:
YAGO: without training:
{
"model": "RecurrencyBaseline",
"train_flag": false,
"data": "tkgl-yago",
"run": 1,
"seed": 1,
"mrr": 0.8588335514068604,
"hits10": 0.9259546399116516,
"test_time": 175.950186,
"tot_train_val_time": 190.072089
}
with training:
{
"model": "RecurrencyBaseline",
"train_flag": "True",
"data": "tkgl-yago",
"run": 1,
"seed": 1,
"mrr": 0.9091410040855408,
"hits10": 0.9302738904953003,
"test_time": 104.039734,
"tot_train_val_time": 922.78976
}
POLECAT mini:
{
"model": "RecurrencyBaseline",
"train_flag": false,
"data": "tkgl-polecat",
"run": 1,
"seed": 1,
"mrr": 0.14627420902252197,
"hits10": 0.21407249569892883,
"test_time": 38.191202,
"tot_train_val_time": 44.85462
},
{
"model": "RecurrencyBaseline",
"train_flag": "True",
"data": "tkgl-polecat",
"run": 1,
"seed": 1,
"mrr": 0.1447719931602478,
"hits10": 0.21252413094043732,
"test_time": 19.861222,
"tot_train_val_time": 210.635383
}
]