QuantML-Github / QuantML

量化机器学习/深度学习模型(Model Zoo);Alpha 因子(Factor Zoo); 量化资源以及相关论文代码
186 stars 37 forks source link

About the results in model_zoo #2

Open Tingji2419 opened 2 months ago

Tingji2419 commented 2 months ago

Thanks for your great work on the reproduciton of QLIB benchmark. But I have a little questions about the results.

First, about the metrics. The values in the table are much higher than those in the original QLIB benchmark (https://github.com/microsoft/qlib/tree/main/examples/benchmarks), e.g. the IC of Transformer model in Alpha360 dataset is 0.08938, which is greater than 0.0114 in the original QLIB benchmark.

Second, about fairness of the training process. Simple LSTMs and Transformers work better than more complex sota models such as Informer and PatchTST. Can you provide details and setup of the data preprocessing and training?

Looking forward to the reply.

QuantML-Github commented 1 month ago

Hello

For the first question, I guess the main difference comes from the stock universe and time period, in my case, I use all a-share and the test period is 2021-2023.

dataset for all the ts models:

dataset: class: TSDatasetH module_path: qlib.data.dataset kwargs: handler: class: Alpha158 module_path: qlib.contrib.data.handler kwargs: *data_handler_config segments: train: [2016-01-01, 2019-12-31] valid: [2020-01-01, 2020-12-31] test: [2021-01-01, 2023-12-31] step_len: 60