Bug fix: MultiOutSizeLinear masks outputs to 0 when dimension of parameter >1 and dim * patch_size is not present in out_feat_size. - Githubissues

SalesforceAIResearch / uni2ts

[ICML2024] Unified Training of Universal Time Series Forecasting Transformers

Apache License 2.0

612 stars 49 forks source link

Bug fix: MultiOutSizeLinear masks outputs to 0 when dimension of parameter >1 and dim * patch_size is not present in out_feat_size. #46

Closed gorold closed 1 month ago

gorold commented 1 month ago

This PR fixes a major bug for the MultiOutSizeLinear class.

When dim > 1 in the following code snippet, https://github.com/SalesforceAIResearch/uni2ts/blob/c4d75a3053c14e27ac4d875d7b31987abc1090a1/src/uni2ts/distribution/_base.py#L74-L85 it causes for feat_size in self.out_features_ls be dim * patch_size, whereas out_feat_size contains the patch sizes. This means that certain patch sizes could be ignored. https://github.com/SalesforceAIResearch/uni2ts/blob/c4d75a3053c14e27ac4d875d7b31987abc1090a1/src/uni2ts/module/ts_embed.py#L172-L178

For the current configuration, this means that outputs with patch sizes 8 and 16 are always masked to 0.

Closes #44

gorold commented 1 month ago

lgtm :)