search

yfzhang114 / OneNet

This is an official PyTorch implementation of the NeurIPS 2023 paper 《OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling》
81 stars 14 forks source link

upload files #5

Open zhangtt2024 opened 2 months ago

zhangtt2024 commented 2 months ago

Your work is excellent and very helpful to me. Can you upload the MOE and Average files in the ensembling method?

yfzhang114 commented 2 months ago

For the average method, you can simply aggregate the results from different experts and compute the average.

Regarding the MOE (Mixture of Experts), the implementation is akin to gating. The main difference lies in the calculation of the gating weights. For gating, it's calculated as follows: $( h = \mathbf{W}_{\text{Concat}}([\tilde{y}_1, \tilde{y}_2]) + \mathbf{b} ) with ( w_1, w_2 = \text{softmax}(h) ).$ Whereas for MOE, it's computed as $( h = \mathbf{W} \mathbf{x} + \mathbf{b} ) with ( w_1, w_2 = \text{softmax}(h) ).$