jankrepl / deepdow

Portfolio optimization with deep learning.
https://deepdow.readthedocs.io
Apache License 2.0
874 stars 136 forks source link

Set callbacks to be based on validation loss (not training loss) #103

Closed turmeric-blend closed 3 years ago

turmeric-blend commented 3 years ago

This is my current setup for a run, EarlyStoppingCallback and ModelCheckpointCallback seems to be based on training loss. How to switch for callbacks to be based on validation loss?

losses = {'MeanReturn': MeanReturns(),
      'CumulativeReturn': CumulativeReturn(),
      'SharpeRatio': SharpeRatio(), 
      'SortinoRatio': SortinoRatio(),}

run = Run(model,
          losses['SharpeRatio'],
          dataloader_train,
          val_dataloaders={'train': dataloader_train,
                           'valid': dataloader_valid},
          metrics = {'MeanReturn': losses['MeanReturn'], 
                     'CumulativeReturn': losses['CumulativeReturn'], 
                 'SharpeRatio': losses['SharpeRatio'], 
                 'SortinoRatio': losses['SortinoRatio']},
          optimizer=optimizer,
          callbacks=[EarlyStoppingCallback(dataloader_name='valid', 
                               metric_name='loss',
                       patience=patience), 
                 ModelCheckpointCallback(folder_path=saved_model_folder, 
                         dataloader_name='valid', 
                         metric_name='loss'), 
             TensorBoardCallback(log_dir=tensorboard_path, 
                     log_benchmarks=True)],
      device=device)
turmeric-blend commented 3 years ago

I realised I read tensorboard wrongly