jdb78
commented
3 years ago You have to explicitly allow unknown categories (see dropout_categoricals in https://pytorch-forecasting.readthedocs.io/en/latest/api/pytorch_forecasting.data.timeseries.TimeSeriesDataSet.html#pytorch_forecasting.data.timeseries.TimeSeriesDataSet). Would be a great PR to improve documentation. Essentially, this sets the embedding to a zero-vector. However, if you have a lot of categories that are not in the training set, it is questionable if you want to include the variable for training.
I can think of three improvements to the current approach. Feel invited to submit a PR
- As you mentioned, one could simply take the average of all known embeddings - to my knowledge, this is a very uncommon approach in deep learning.
- Opposite to what the name of the parameter implies, no random dropout of categoricals in performed. Implementing this could improve the interpretation of the zeroed embeddings. This approach is likely to result in very similar behaviour as option 1.
- You could pre-train the network with a decoder network that predicts masked features. This is an architecture specific approach and might even boost final accuracy. A good example is TabNet.
diditforlulz273
Looks like I've found a bug/unexpected behavior. I'm making a prediction on a dataset with time-based features marked as 'categorical', namely month alongside day and year. The start of the dataset is 2020-01-01, and the end is 2020-08-30. The date is parsed into 'year, 'month', and 'day' columns for each row. Depending on the last dataset record's date(if I cut it for some reason), pytorch-forecasting throws an error that looks like:
Traceback: File "XXX/venv/lib/python3.8/site-packages/pytorchforecasting/data/encoders.py", line 105, in
encoded = [self.classes [v] for v in y]
KeyError: '8'
I've made some experiments/stack traces and this is always the case when you, for instance, have this month(8, August) in the full set but don't have it in your training set - for the reason that your max_prediction_length is bigger than 31 (day) or you have a combination of the last date and max_predlength like 2020-08-10 and 20, so the last date of training set will be ~2020-07-20 and it won't have '8' month inside. In this case, going back to the code line provided in traceback, you have this value(8) in np.unique(y) (iterator), BUT in **self.classes** you don't.
Seems like self.classes_ is created based on the training set only, and when you try to invoke TimeSeriesDataSet.from_dataset(trainigset, fullset, .....) you get this error for any additional categorical values that might have appeared in the full dataset.
This logic makes it practically hard to be used on any type of date/time categorically encoded datasets.
Shouldn't any previously unseen categorical value be put into the special 'average' bin and treated as the average of all the known categories? As far as I remember, LightGBM exhibits this behavior for any new categorical values.