No dropout in NHITS 😧

[hiandersson]

Describe the bug Trying to use dropout in NHITS results in not implemented error

To Reproduce Set dropout_prob_theta to above 0 when construing the NHITS model raise NotImplementedError("dropout")

Expected behavior Dropout should work in NHITS

Additional context Dropout is a very important regularisation method to avoid overfitting. Not having it is a major blocker for me. Can I help? I see the line is commented out, is there any reason why?

if self.dropout_prob > 0:
    raise NotImplementedError("dropout")
    # hidden_layers.append(nn.Dropout(p=self.dropout_prob))

[hiandersson]

@kdgutier any thoughts on this? many thanks for the hard work on this lib

[kdgutier]

Hey @hiandersson,

Regarding the dropout regularization, we followed Oreshkin's recommendation to not include it. See NBEATS paper Section 3.4 page 5: https://arxiv.org/pdf/1905.10437.pdf

There are alternative regularization methods included in the NHITS:

• Early stopping.
• Expressivity ratios in the interpolation.
• Network's hidden units and depth.
• You can additionally create an ensemble to continue reducing the variance of the method's predictions.

If you still need dropout in the NHITS. It should be a quick addition in this notebook: https://github.com/Nixtla/neuralforecast/blob/main/nbs/models.nhits.ipynb

Let me know.

[hiandersson]

Many thanks, I will check it out and do some tests, I will get back with feedback.

[hiandersson]

Many thanks @kdgutier