scverse / scvi-tools

Deep probabilistic analysis of single-cell and spatial omics data
http://scvi-tools.org/
BSD 3-Clause "New" or "Revised" License
1.26k stars 356 forks source link

Model hyperparameter tuning with autotune fails on colab tutorial #3055

Closed alexclarke7 closed 2 days ago

alexclarke7 commented 4 days ago

Model hyperparameter tuning with scVI tutorial fails to run on colab tutorial page (https://docs.scvi-tools.org/en/1.0.0/tutorials/notebooks/autotune_scvi.html).

!pip install --quiet hyperopt
!pip install --quiet "ray[tune]"
!pip install --quiet scvi-colab
from scvi_colab import install

install()

model_cls = scvi.model.SCVI
model_cls.setup_anndata(adata)

scvi_tuner = autotune.ModelTuner(model_cls)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[180], line 1
----> 1 tuner = autotune.ModelTuner(model_cls)

AttributeError: module 'scvi.autotune' has no attribute 'ModelTuner'

Versions:

VERSION

1.2.0

canergen commented 4 days ago

Hi, could you please refer to the tutorial for 1.2.0: https://docs.scvi-tools.org/en/1.2.0/tutorials/notebooks/tuning/autotune_scvi.html.

Best!

alexclarke7 commented 4 days ago

Thank you.

I am trying to discover the tunable parameters for a model according to the tutorial as below:

'Our main entry point to the autotune module is the ModelTuner class, a wrapper around ray.tune.Tuner with additional functionality specific to scvi-tools. We can define a new ModelTuner by providing it with our model class.

ModelTuner will register all tunable hyperparameters in SCVI – these can be viewed by calling info(). By default, this method will display three tables:

Tunable hyperparameters: The names of hyperparameters that can be tuned, their default values, and the internal classes they are defined in. Available metrics: The metrics that can be used to evaluate the performance of the model. One of these must be provided when running the tuner. Default search space: The default search space for the model class, which will be used if no search space is provided by the user.' ###

tuner = autotune.ModelTuner(model_cls)
tuner.info()

However this leads to the error above.

Thanks again for your help with this.

canergen commented 4 days ago

You can now tune all model parameters. It is not required anymore to define a parameter as tunable in the model. The tutorial that I shared highlights how to set up the search space.

alexclarke7 commented 4 days ago

Thanks

previously tuner.info() had created a nice table of default values for a given model, which was helpful to choose the search space, which is mentioned in the tutorial page you linked, but I can't access it. Is that paragraph of the tutorial still correct?

canergen commented 4 days ago

This function is not supported anymore and now that all arguments are accessible it doesn't make sense anymore as the table will get quite large. We prefer exposing every parameter to ray.tune. @ori-kron-wis Can you remove this sentence from the tutorial?