[Question] Cannot overfit TCN and Transformer model

[leyp1]

I have been trying overfit the darts models on my dataset by only training on one sample of the dataset. This worked for both the RNN model and the NBEATS model, however the TCN and Transformer models have not followed in the same behavoir. I am training the models using the fit() function using a validation/training split based on a set date. TCN Model on one datasample Transformer Model on one datasample

What I expected: (RNN Model)

System:

• Python version: [e.g. 3.7]

• darts version [e.g. 0.14.0]

  I am uncertain if I used the package incorrectly or if it could be another issue here, I would be happy about any response. If there is any more information you need I would happily provide this.

[dennisbader]

TransformerModel and TCNModel have a default dropout > 0., whereas RNNModel has a default of 0..

I assume that setting the dropout to 0. for the two models should result in higher overfitting

[leyp1]

Thank you for your quick answer. I get an "RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation" error when setting dropout to 0 for the TCN model, but when setting the dropout to a very small value I indeed manage to overfit it.

However the transformer model still has trouble: I am only training it on one time series so I don't think it could be a capacity problem. But I can't really explain the behavior ...

[dennisbader]

You need to upgrade your darts version, then it should work (the TCN issue)