Do you have results of Simple-HGN on the GTN-datasets?

[1049451037]

Here https://github.com/THUDM/HGB/tree/master/NC/GTN

[amblee0306]

Sorry if I wasn't clear, what I meant was the results in this repo are evaluated on the HGB dataset. I would like to know if you have tried the model proposed (Simple-HGN) on the dataset that was initially proposed by GTN. I assumed yes cause in the README, it was written "Moreover, we also have an implementation of Simple-HGN in [cogdl]"(https://github.com/THUDM/cogdl/tree/master/examples/simple_hgn).

I checked and ran the cogdl implementation (it uses the dataset by GTN) and Simple-HGN performed quite badly in that situation as compared to GCN / GAT. Is there a reason why?

[1049451037]

If you see the link I provide, you can see the experiments absolutely from GTN.

[amblee0306]

what about your model but on GTN's dataset?

[1049451037]

In HGB benchmark, the IMDB dataset is just from GTN. If you want to get the results of all datasets of GTN, you can just replace the GAT model in the link I provided with Simple-HGN model, which are nearly same code in our repo.

[amblee0306]

Okays got it, thanks!

[themaigod]

@amblee0306 I also get performed quite badly for Simple-HGN in GTN datasets. I revised the data_loader to load GTN dataset. Is it much improved by the authors suggestions?

[amblee0306]

@themaigod I can achieve reasonable results using the authors' suggestions. You can see our reported results in https://openreview.net/forum?id=VyfEv6EjKR

[themaigod]

@amblee0306 Thanks and Congrates! I am wondering that you choose the cogdl model or from the current respository provided, since I find cogdl model doesn't do transformations for node feature in different node types respectively. Maybe I will try both and select the better one.

[amblee0306]

@themaigod I can't remember exactly. But I remembered trying both ways as well. All the best!

[themaigod]

@amblee0306 Thanks. I have verified them. It seems that doing transformations for node feature in different node types respectively leads to a terrible result. For directly using cogdl model, the result is much more resonable, though 1 in 10 seeds achieve a quite bad result.