Closed kevkid closed 3 years ago
jdb78
commented
3 years ago The reason seems to be missing input. encoder_cont and encoder_cat are both empty. I suggest to specify at least the target to be in time_varying_unknown_reals. You probably also want to add, at least, the time_idx there.
kevkid
commented
3 years ago @jdb78 Thank you for that hint. I apologize I am new to time series data. I did want to understand this structure a little further so I will ask you a few questions that will clear things up for me. Why would target be in time_varying_unknown_reals when there is a target parameter when setting up the dataset? Also time_varying_unknown_reals vs time_varying_known_reals in what context would we use one over the other? One final question: Where would my time_index go? I just made a column that for each row, this column increments by 1 and set the time_idx= 'time_idx'.
EDIT: I tried it again with your suggestions and things seem to be working. Although I am curious, does this look like the right approach?
# define dataset
max_encode_length = 36
max_prediction_length = 6
training_cutoff = "2019-12-31" # day for cutoff
training = TimeSeriesDataSet(
data[lambda x: x.Date <= training_cutoff],
time_idx= 'time_idx',
target= 'Close',
group_ids=['groups'],
#min_encoder_length = 1,
max_encoder_length=max_encode_length,
max_prediction_length=max_prediction_length,
allow_missings=True,
#static_categoricals=[ ... ],
#static_reals=[ ... ],
#time_varying_known_categoricals=[ ... ],
time_varying_known_reals=['time_idx' ],#, 'High', 'Low', 'Volume' ],
#time_varying_unknown_categoricals=[ ... ],
time_varying_unknown_reals=[ 'Close','Open'],
)
validation = TimeSeriesDataSet.from_dataset(training, data, min_prediction_idx=training.index.time.max() + 1, stop_randomization=True)
batch_size = 32
train_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=8)
val_dataloader = validation.to_dataloader(train=False, batch_size=batch_size, num_workers=8)
early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-4, patience=1, verbose=False, mode="min")
lr_logger = LearningRateMonitor()
trainer = pl.Trainer(
max_epochs=100,
gpus=None,
gradient_clip_val=0.1,
limit_train_batches=30,
callbacks=[lr_logger, early_stop_callback],
)
tft = TemporalFusionTransformer.from_dataset(
training,
learning_rate=0.01,
hidden_size=16,
attention_head_size=1,
dropout=0.1,
hidden_continuous_size=16,
output_size=7,
loss=QuantileLoss(),
log_interval=2,
reduce_on_plateau_patience=4
)
tft.cpu()
print(f"Number of parameters in network: {tft.size()/1e3:.1f}k")Yes :) Looks good to me.
paapu88
commented
3 years ago Thanks for this issue! Sorted out my problems also. Markus
LiuNull
commented
2 years ago Thanks for this issue! Sorted out my problems too.
JustusMzB
commented
1 year ago Using the time id as a time varying known real has led to a special kind of overfitting for me: The TFT starts learning the values to appropriate time id's by heart. Therefore I can't recommend this workaround.
Expected behavior
I am trying to predict future closing prices from past opening prices of bitcoin. I have downloaded a dataset for all open and close prices for the past 6 years. The format of the data is:
Here is my code:
Here is what I get as output:
I thought that it had something to do with the train/valid loader so I just looped over one to see:
and got:
but my batch looks like:
Any idea why this would occur? I am not sure if my dataloader is in the wrong format?