search

midas-research / sthan-sr-aaai

98 stars 32 forks source link

Experimental results do not match #7

Open kongxin-come opened 2 years ago

wmmxk commented 2 years ago

Could you describe how different the results you got? E.g. what metric you used to compare.

Droliven commented 1 year ago

Hello, author. I tried to reproduce your code, but the results of the experiments are quite different from the results given in the paper. Can you explain the construction of the hypergraph and the implementation of the parts of the hypergraph Attention?

Could you please share your source code? How do you get the 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy'?

280777510 commented 1 year ago

Could you please share your source code? How do you get the 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy'?

Could you please share your source code? How do you get the 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy'?

280777510 commented 1 year ago

Hello, author. I tried to reproduce your code, but the results of the experiments are quite different from the results given in the paper. Can you explain the construction of the hypergraph and the implementation of the parts of the hypergraph Attention?

Could you please share your source code? How do you get the 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy'?

Harryx2019 commented 1 year ago

Hello, author. I tried to reproduce your code, but the results of the experiments are quite different from the results given in the paper. Can you explain the construction of the hypergraph and the implementation of the parts of the hypergraph Attention?

Hello. I have also recently tried to reproduce this code, and I find the same problem with you. It seems that the result in the paper can't realized. I also tried the timing cross-validation methods but failed at the same time. Otherwise, i find that during the period of 2016 this code can achieve better result in the back testing, however, once it applied on the period of 2017, it is terrible. So, i sincerely wonder if you are still doing research on this topic and i want to ask for your help!

mumu-learn commented 1 year ago

Could you please share your source code? How do you get the 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy'?Thanks @280777510

yezeping commented 1 year ago

In the process of reproducing, I encountered all the above problems. Especially when the model was backtested on the 2017 data, the effect was very poor. I suspect that the construction of hpy_attention is wrong. I wonder if anyone can help me?

mumu-learn commented 1 year ago

@yezeping Could you please share the relevant data, including the data of 'nasdaq.npy', 'hypergraph_nyse.npy' and 'tse.npy', thank you.

M1stF0rest commented 8 months ago

Hello, author. I tried to reproduce your code, but the results of the experiments are quite different from the results given in the paper. Can you explain the construction of the hypergraph and the implementation of the parts of the hypergraph Attention?

Hello. I have also recently tried to reproduce this code, and I find the same problem with you. It seems that the result in the paper can't realized. I also tried the timing cross-validation methods but failed at the same time. Otherwise, i find that during the period of 2016 this code can achieve better result in the back testing, however, once it applied on the period of 2017, it is terrible. So, i sincerely wonder if you are still doing research on this topic and i want to ask for your help!

Hi, I just reproduced the code as well. My results show that on the test set, the trained model even keeps losing money following on their evaluation criteria... I wonder whether it is because the period the author used for training and testing is too far away from each other(They have a time gap for nearly three years and the market could change a lot by then) Moreover, the authors were ambiguous about how they construct the hypergraph & what are the edge features used for hyper graph attention, so I just self-implemented them based on my understanding, and maybe that could have some influence on the result. Anyway, if you are still interested in this topic, feel free to reach out to me!