Recommendations for scaling real features with DeepAR?

[pbruneau]

I initially found the topic below better suited for a discussion, but I have the impression discussions are disregarded compared to issues, and the topic is quite crucial IMHO.

Discussed in https://github.com/awslabs/gluon-ts/discussions/1813

^{Originally posted by **pbruneau** December 3, 2021} Reading the DeepAR paper (Salinas et al. 2020), I understood that each target time series is scaled for training/prediction by the model, and the scale is inverted for computing the actual outputs. Do similar scaling operations also apply to the additional real features provided to the model (FEAT_STATIC_REAL and FEAT_DYNAMIC_REAL)? Are there recommendations regarding the range of these features (standardization, unit scaling, other)? Thanks by advances for the tips, and with regards

[kashif]

@pbruneau since these features come with the data or externally, I typically scale them in the data prep. stage rather than on the model side... (which I suppose also applies to the target... but I guess it's more convenient to scale the target in the model...).

[StatMixedML]

@pbruneau @kashif A somewhat related issue is the scaling of the response/target value in Deep-Learning models. Citing from this paper illustrates the problem

To allow for training deep learning models with time series data, the data must be first appropriately normalized. Perhaps the most widely used normalization scheme for time series when using DL is the z-score normalization, i.e., subtracting the mean value of the data and dividing by their standard deviation. However, z-score normalization is unable to efficiently handle non-stationary time series, since the statistics used for the normalization are fixed both during the training and inference.

I find this to be a very important issue that is not being addressed properly in most models. The paper suggests to learn the scaling during model training directly

To overcome these limitations, we propose a Deep Adaptive Input Normalization (DAIN) layer that is capable of a) learning how the data should be normalized and b) adaptively changing the applied normalization scheme during inference, according to the distribution of the measurements of the current time series, allowing for effectively handling non-stationary and multimodal data.

The approach is quite nice, even though I am not sure how to inverse the transformation to arrive at the initial scale. The problem lies with the activation functions. Is there a way to inverse the layer operation for arbitrary layers and activation functions?

[alisha-metkari-aera]

@kashif , facing issue while using dynamic real features and static cat features with deepAR, getting error at the line next(prediction).quantile(0.5)

prediction = predictor.predict(train_ds)
horizon_output = pd.DataFrame()
horizon_output['Forecast'] = next(prediction).quantile(0.5)

Error is : ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 792 and the array at index 2 has size 768 Can you help me to get prediction when we use features with model.