Training a TemporalFusionTransformer batch-wise

[TomSteenbergen]

• PyTorch-Forecasting version: 0.8.3
• PyTorch-Lightning version: 1.1.8
• Python version: 3.8.1
• Operating System: macOS Big Sur 11.2.1

First of all, thanks for the amazing work on this package!

I have a question on batch-wise training. I'd like to use the Temporal Fusion Transformer model on a very large data set. However, I cannot load this data set in-memory due to its size. Therefore, I would like to be able to train the TFT model batch-wise. I.e. fetch a single batch from the database, perform all necessary preprocessing on this batch, and perform one feedforward and backward pass. There should be only one batch in-memory at any time.

I couldn't find a clear example in the docs. However, in the docs for TimeSeriesDataset, I found the following note:

Large datasets:

Currently the class is limited to in-memory operations (that can be sped up by an existing installation of numba). If you have extremely large data, however, you can pass prefitted encoders and and scalers to it and a subset of sequences to the class to construct a valid dataset (plus, likely the EncoderNormalizer should be used to normalize targets). when fitting a network, you would then to create a custom DataLoader that rotates through the datasets. There is currently no in-built methods to do this.

So there is currently no in-built method to do this.

• First question: Is this anywhere on the roadmap for the short term?

• Second question: If not, would something like the code snippet below work? Can I repeatedly call TimeSeriesDataSet and trainer.fit, or should I use a custom loss function and optimizer in a loop like this? Also, besides encoders and scalers, are there any other things that will need to prefitted before passing to TimeSeriesDataset? Lastly: is there an overall better approach to this problem?

  
  db_conn = ...  # Database connection.

validation_data = db_conn.fetch_all(...) # Fetch all data of validation set. validation_dataset = TimeSeriesDataset(validation_data, ...) validation_dataloader = validation_dataset.to_dataloader(...)

trainer = Trainer(...) tft = TemporalFusionTransformer(...)

for batch in db_conn.fetch_batches(...): # Fetch training batch from database using some generator. train_dataset = TimeSeriesDataSet(batch, ...) train_dataloader = train_dataset.to_dataloader(...) trainer.fit(tft, train_dataloader, validation_dataloader)

Many thanks in advance!

[jdb78]

I am not planning for an out of memory version very soonish because the vast majority of datasets should fit into memory (particularly if you use a cloud server). Indeed, only encoders and scalers need to be prefit. Your approach looks reasonable to me. A more integrated solution could be to wrap the multiple datasets in one dataset. Could be an interesting PR.