search

tufts-ml / KCN

Kriging Convolutional Networks (KCN) in Tensoflow
38 stars 17 forks source link

Questions about the “Testing” part on kcn_base.py #2

Closed jlidw closed 2 years ago

jlidw commented 2 years ago

Hi, when reproducing the results, I have some questions.

In the testing stage of KCN, (y.shape[0] - Ntrain) nodes are used as testing nodes, but these testing nodes should be at the end of the dataset, right? So in line 191, it should be test_ind = range(Ntrain, y.shape[0]) instead of test_ind = range(num_test), otherwise, the model will take the training nodes at the beginning of the dataset for testing.

Also, in line 192, when doing the evaluation of testing data, I think the input should be "y_train_val" instead of "y", that is, the true labels of testing nodes should not be used as known info.

It seems the current codes are using training nodes for testing, and the true labels of testing nodes are wrongly added as the known info, which is not fair during the evaluation.

Looking forward to your reply, thanks very much.

截屏2022-01-23 下午3 52 54
JumboCasher commented 2 years ago

Thanks for your valuable feedback.

As this repo is originally created for kcn_sage (kcn_sage is our best model), please refer to kcn/kcn_sage.py as our final implementation. In that file, the two issues do not exist. https://github.com/tufts-ml/KCN/blob/33d08ff8cc8a966cb1ce0391150aa9a0438fe59d/kcn/kcn_sage.py#L198

Initially, the repo also included kcn_base just for completeness, but the file is outdated. We will update the repo soon.