search

scverse / scvi-tools

Deep probabilistic analysis of single-cell and spatial omics data
http://scvi-tools.org/
BSD 3-Clause "New" or "Revised" License
1.21k stars 344 forks source link

How to tune hyperparameters? #302

Closed QqQss closed 5 years ago

QqQss commented 5 years ago

Hi, guys, thanks for your great tools!

I'm a little confused in hyper-parameter tuning. In most case, we could select a part of cells as test_set, then compute likelihood error constantly. According to the likelihood change we could know whether the number of epoch is enough and which comb of hyper-parameters will get the minimum error in the last epoch. Is it right?

But in some of your tutorials, e.g. 'harmonization' and 'interaction with scanpy'. You choose all of cells as training set, and then I find that it cannot get likelihood error info anymore! Maybe cannot compute training likelihood alone? (i.e. set frequency=5 and train_size=1.0, then will return an zero division error). In that case, if we cannot trace likelihood change history, how could we know whether the number of epoch is enough? and how to tune the hyperparameters?

romain-lopez commented 5 years ago

Hi,

We are working on automatic hyperparameters tuning. In the meantime, you can just train with less cells to choose hyperparameters (train_size < 1) and then re-train with optimal parameters with the training_set.

QqQss commented 5 years ago

That's a solution! Thanks for your reply :)

jeff-regier commented 5 years ago

You can check out PR #299 to follow along with our work on automatic hyperparameter tuning