Closed namsaraeva closed 3 months ago
Note to self: look at AdamW as optimizer
AdamW's weight decay default value is 1e-2, but I am not sure if I should add it as self.hparams["weight_decay"] = 10 ** -2
too
It should be possible to pass on a column name instead of column index for target_col (the col with float timepoint targets) in HDF5SingleCellDatasetRegression
@sophiamaedler should we stick to the VGG architecture with 11 or 16 layers? for MLP: config A or B?
UPD: stick to the default architecture for now
Consider adding HuberLoss instead of MSE, MAE
Consider adding cross validation or grid search to test different settings for delta in Huber loss to get an optimal value.
"Huber loss will clip gradients to delta for residual (abs) values larger than delta. You want that when some part of your data points poorly fit the model and you would like to limit their influence." - we probably want to limit the influence of phenotypes that are similar to previously seen phenotypes? @sophiamaedler
Current limitation: scatterplot lims when training. We have to somehow pass the min and max values of dataset to the Callback module.
Also implement a flexible Callback module that can be re-used.
A draft of the simple VGG-based neural network with a single output instead of the last classification layer.