moraieu / query-selector

LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION
Apache License 2.0
74 stars 19 forks source link

A quick question about the linear layer #2

Closed kpmokpmo closed 2 years ago

kpmokpmo commented 2 years ago

Hi, just a quick question.

I didnt't have a chance to run the code, so I assume the input data have a shape like B,L,C or B,C,L. Line279: self.enc_input_fc = Linear(input_size, dim_val). I think “input_size” means the length while “dim_val” is somewhat related to embedding dimensions. And the shape of the tensor turned into a square which doesn't make sense to me. Is that a typo or I misunderstand something? Many thanks!

j-klm commented 2 years ago

Input_size is the length of an individual vector in the input sequence. So for univariate forecasting it is set to 1 and for multivariate forecasting on ETT it's 7. So incoming batch has shape B, L, input_size. After passing through enc_input_fc Linear layer the shape becomes B, L, dim_val as it should be.