ModelTuner.fit Returns error: The `RunConfig(local_dir)` argument is deprecated.

[osmanmerdan]

I was trying out autotune. I ran ModelTuner.fit, and I encountered the following error. It may be related to changes in the ray package: https://docs.ray.io/en/latest/train/user-guides/persistent-storage.html#setting-the-local-staging-directory

model_cls = scvi.model.SCVI
model_cls.setup_anndata(adata=adata_scvi, batch_key=batch_key, layer="counts") 
scvi_tuner = autotune.ModelTuner(model_cls)
#scvi_tuner.info()

search_space = {
    "n_hidden": tune.choice([92, 128, 192, 256]),
    "n_latent": tune.choice([10, 20, 30, 40, 50, 60]),
    "n_layers": tune.choice([1, 2, 3]),
    "lr": tune.loguniform(1e-4, 1e-2),
    "gene_likelihood": tune.choice(["nb", "zinb"])}

results = scvi_tuner.fit(
    adata_scvi,
    metric="validation_loss",
    search_space=search_space,
    num_samples=5,
    max_epochs=100,
    resources={"cpu": 10, "gpu": 1},
    #logging_dir='/content/logging'
)

[/usr/local/lib/python3.10/dist-packages/scvi/autotune/_tuner.py](https://localhost:8080/#) in fit(self, adata, **kwargs)
    113             A dataclass containing the results of the tuning experiment.
    114         """
--> 115         tuner, config = self._manager._get_tuner(adata, **kwargs)
    116         results = tuner.fit()
    117         return self._manager._get_analysis(results, config)

[/usr/local/lib/python3.10/dist-packages/scvi/autotune/_manager.py](https://localhost:8080/#) in _get_tuner(self, adata, metric, additional_metrics, search_space, model_kwargs, train_kwargs, use_defaults, num_samples, max_epochs, scheduler, scheduler_kwargs, searcher, searcher_kwargs, reporter, resources, seed, experiment_name, logging_dir, monitor_device_stats)
    556             num_samples=num_samples,
    557         )
--> 558         run_config = air.config.RunConfig(
    559             name=_experiment_name,
    560             local_dir=_logging_dir,

/usr/local/lib/python3.10/dist-packages/ray/air/config.py in __init__(self, name, storage_path, storage_filesystem, failure_config, checkpoint_config, sync_config, verbose, stop, callbacks, progress_reporter, log_to_file, local_dir)

[/usr/local/lib/python3.10/dist-packages/ray/air/config.py](https://localhost:8080/#) in __post_init__(self)
    663 
    664     def __repr__(self):
--> 665         from ray.train import SyncConfig
    666 
    667         return _repr_dataclass(

DeprecationWarning: The `RunConfig(local_dir)` argument is deprecated. You should set the `RunConfig(storage_path)` instead.See the docs: https://docs.ray.io/en/latest/train/user-guides/persistent-storage.html#setting-the-local-staging-directory

Versions:

ray: 2.32.0 scvi:1.1.5

VERSION

[canergen]

This is a deprecation warning and not an error, so the results will all be correct. In our current main branch, this deprecation warning will not be raised and it contains a fully remodelled backend for autotuning.

[Sporgelum]

Hi @canergen. I followed the instructions and got the newest scvi-tools from the main branch, but I am not able to solve it... The goal is the following code:

# Check if 'raw_counts' is a valid key
if 'raw_counts' not in adata_horses.layers:
    scvi.model.SCVI.setup_anndata(adata_horses, batch_key="bio_batch")
else:
    scvi.model.SCVI.setup_anndata(adata_horses, layer="raw_counts", batch_key="bio_batch")

import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
torch.set_float32_matmul_precision("high")
save_dir = "/data/projects/p918_Horse_Dendritic_Cells_Comparison/Analysis_STARsolo_samples/Load_and_Integrate/adatas/autotune"
scvi.settings.logging_dir = save_dir

# Initialize Ray
ray.init(log_to_driver=False)

search_space = {
    "n_hidden": tune.choice([128, 256, 512]),
    "n_layers": tune.choice([2, 3, 5]),
    "n_latent": tune.choice([10, 20, 30, 40, 50]),
    "batch_size": tune.choice([64, 128, 256, 512]),
    "lr": tune.loguniform(1e-4, 1e-3, 1e-2),
    # "early_stopping": tune.choice([True]),
    # "early_stopping_monitor": tune.choice(["elbo_validation"]),
    # "early_stopping_min_delta": tune.choice([0.00]),
    # "early_stopping_patience": tune.choice([20])
}   

# Define the fixed model parameters using model_kwargs
model_kwargs = {
    "dropout_rate": 0.25,
    "encode_covariates": True,
    "deeply_inject_covariates": False,
    "use_layer_norm": "both",
    "use_batch_norm": "none",
    # Any other fixed parameters can be included here.
}
# Define the fixed training parameters using train_kwargs
train_kwargs = {
    "early_stopping": True,
    "early_stopping_monitor": "elbo_validation",
    "early_stopping_min_delta": 0.00,
    "early_stopping_patience": 10,
   # "accelerator": 'gpu',  # Use GPU accelerator
    # "devices": num_devices,  # Use the number of available CUDA devices
}
# Initialize the ModelTuner
scvi_tuner = autotune.ModelTuner(scvi.model.SCVI)
print("All tunable features in SCVI")
scvi_tuner.info()

# Run the tuner with the specified parameters
results = scvi_tuner.fit(
    adata_horses,
    metric='validation_loss',
    search_space=search_space,
    num_samples=100,
    max_epochs=200,
    resources={'cpu': 32, 'gpu': 2},
    #logging_dir=save_dir,  
    storage_path=save_dir,  # Using storage_path instead of logging_dir
    model_kwargs=model_kwargs,
    train_kwargs=train_kwargs
)

# Shutdown Ray
ray.shutdown()

I seems to work until the point I get the same as @osmanmerdan

{
    "name": "DeprecationWarning",
    "message": "The `RunConfig(local_dir)` argument is deprecated. You should set the `RunConfig(storage_path)` instead.See the docs: https://docs.ray.io/en/latest/train/user-guides/persistent-storage.html#setting-the-local-staging-directory",
    "stack": "---------------------------------------------------------------------------
DeprecationWarning                        Traceback (most recent call last)
Cell In[34], line 55
     51 scvi_tuner.info()
     54 # Run the tuner with the specified parameters
---> 55 results = scvi_tuner.fit(
     56     adata_horses,
     57     metric='validation_loss',
     58     search_space=search_space,
     59     num_samples=100,
     60     max_epochs=200,
     61     resources={'cpu': 32, 'gpu': 2},
     62     #logging_dir=save_dir,  
     63     #storage_path=save_dir,  # Using storage_path instead of logging_dir
     64     model_kwargs=model_kwargs,
     65     train_kwargs=train_kwargs
     66 )
     68 # Shutdown Ray
     69 ray.shutdown()

File /data/users/mbotos/Environments/Porcine_DC_MC_2024_03_18/Raw_Data_Analysis/2024_03_25_Porcine_PBMC_DC_M/lib/python3.9/site-packages/scvi/autotune/_tuner.py:115, in ModelTuner.fit(self, adata, **kwargs)
     33 def fit(self, adata: AnnOrMuData, **kwargs) -> None:
     34     \"\"\"Run a specified hyperparameter sweep for the associated model class.
     35 
     36     Parameters
   (...)
    113         A dataclass containing the results of the tuning experiment.
    114     \"\"\"
--> 115     tuner, config = self._manager._get_tuner(adata, **kwargs)
    116     results = tuner.fit()
    117     return self._manager._get_analysis(results, config)

File /data/...../lib/python3.9/site-packages/scvi/autotune/_manager.py:558, in TunerManager._get_tuner(self, adata, metric, additional_metrics, search_space, model_kwargs, train_kwargs, use_defaults, num_samples, max_epochs, scheduler, scheduler_kwargs, searcher, searcher_kwargs, reporter, resources, seed, experiment_name, logging_dir, monitor_device_stats)
    538 _trainable = self._get_trainable(
    539     adata,
    540     _metrics,
   (...)
    550     monitor_device_stats=monitor_device_stats,
    551 )
    553 tune_config = tune.tune_config.TuneConfig(
    554     scheduler=_scheduler,
    555     search_alg=_searcher,
    556     num_samples=num_samples,
    557 )
--> 558 run_config = air.config.RunConfig(
    559     name=_experiment_name,
    560     local_dir=_logging_dir,
    561     progress_reporter=_reporter,
    562     log_to_file=True,
    563     verbose=1,
    564 )
    565 tuner = tune.Tuner(
    566     trainable=_trainable,
    567     param_space=_search_space,
    568     tune_config=tune_config,
    569     run_config=run_config,
    570 )
    571 config = {
    572     \"metrics\": _metrics,
    573     \"search_space\": _search_space,
    574 }

File <string>:15, in __init__(self, name, storage_path, storage_filesystem, failure_config, checkpoint_config, sync_config, verbose, stop, callbacks, progress_reporter, log_to_file, local_dir)

File /data/....../lib/python3.9/site-packages/ray/air/config.py:665, in RunConfig.__post_init__(self)
    662 from ray.tune.experimental.output import AirVerbosity, get_air_verbosity
    664 if self.local_dir is not None:
--> 665     raise DeprecationWarning(
    666         \"The `RunConfig(local_dir)` argument is deprecated. \"
    667         \"You should set the `RunConfig(storage_path)` instead.\"
    668         \"See the docs: https://docs.ray.io/en/latest/train/user-guides/\"
    669         \"persistent-storage.html#setting-the-local-staging-directory\"
    670     )
    672 if self.storage_path is None:
    673     # TODO(justinvyu): [Deprecated] Remove in 2.30
    674     self.storage_path = DEFAULT_STORAGE_PATH

DeprecationWarning: The `RunConfig(local_dir)` argument is deprecated. You should set the `RunConfig(storage_path)` instead.See the docs: https://docs.ray.io/en/latest/train/user-guides/persistent-storage.html#setting-the-local-staging-directory"
}

[Sporgelum]

Hi @canergen ,

Very quick update, for me it was working if I just went to the scvi/autotune/manager.py and I changed it to

558         run_config = air.config.RunConfig(
559             name=_experiment_name,
560             storage_path=_logging_dir,
561             progress_reporter=_reporter,
562             log_to_file=True,
563             verbose=1,
564         )

[canergen]

Hi, somehow your installation went wrong. /data/...../lib/python3.9/site-packages/scvi/autotune/_manager.py doesn't exist anymore in main. Autotune is fully refactored. Unfortunately, raise DeprecationWarning is not a preferred way to mark deprecations as it throws an exception (my response above was obviously wrong in that regard). I would suggest to otherwise downgrade ray to 2.23: release notes.

[osmanmerdan]

Hi @canergen ,

Very quick update, for me it was working if I just went to the scvi/autotune/manager.py and I changed it to

558         run_config = air.config.RunConfig(
559             name=_experiment_name,
560             storage_path=_logging_dir,
561             progress_reporter=_reporter,
562             log_to_file=True,
563             verbose=1,
564         )

I did the same thing to make it work. I had installed scVI with pip.

[sunhollyjolly]

Hi @canergen ,

Very quick update, for me it was working if I just went to the scvi/autotune/manager.py and I changed it to

558         run_config = air.config.RunConfig(
559             name=_experiment_name,
560             storage_path=_logging_dir,
561             progress_reporter=_reporter,
562             log_to_file=True,
563             verbose=1,
564         )

Had same issue, and this solved the problem.

[canergen]

@ori-kron-wis can you do a patch fix as 1.11.6 as discussed.

[Pinolinoo]

that worked too, is finally running now! thanks so much for your help!

[ori-kron-wis]

1.1.6 is released

[ori-kron-wis]

In addition to the above note for v1.1.6 you will need to use setuptools <=69.5.1