RobbyW551 commented 1 year ago

NOTE: ISSUES ARE NOT FOR CODE HELP - Ask for Help at https://stackoverflow.com

Your issue may already be reported! Also, please search on the issue tracker before creating one.

I'm submitting a ...
- [x] bug report

Issue Description

When Issue Happens Running time-series forecasting example.
Steps To Reproduce
1. Install autopytorch-v0.2 and run time-series forecasting example with user warning displayed.
2. Copy https://github.com/automl/Auto-PyTorch/blob/master/autoPyTorch/configs/forecasting_init_cfgs.json to corresponding auto-pytorch repo in python site-package directory.
3. Run time-series forecasting example with user warning allowed.

Expected Behavior

Traverse 9 initial designs and do evaluation on each, then different random search configurations will be evaluated.

Current Behavior

After the user warning is raised, only a single default configuration is evaluated before random searching begins.

Your Code

same as https://github.com/automl/Auto-PyTorch/blob/master/examples/20_basics/example_time_series_forecasting.py except allowing user warnings.

Error Message

/home/robby/miniconda3/envs/auto-pytorch/lib/python3.8/site-packages/autoPyTorch/optimizer/utils.py:97: UserWarning: Failed to convert {'data_loader:batch_size': 32, 'data_loader:backcast': False, 'data_loader:sample_strategy': 'SeqUniform', 'data_loader:num_batches_per_epoch': 50, 'data_loader:transform_time_features': False, 'lr_scheduler:__choice__': 'ReduceLROnPlateau', 'lr_scheduler:ReduceLROnPlateau:mode': 'max', 'lr_scheduler:ReduceLROnPlateau:factor': 0.5, 'lr_scheduler:ReduceLROnPlateau:patience': 10, 'optimizer:__choice__': 'AdamOptimizer', 'optimizer:AdamOptimizer:lr': 0.001, 'optimizer:AdamOptimizer:weight_decay': 1e-08, 'optimizer:AdamOptimizer:beta1': 0.9, 'optimizer:AdamOptimizer:beta2': 0.999, 'network_init:__choice__': 'XavierInit', 'network_init:XavierInit:bias_strategy': 'Normal', 'target_scaler:scaling_mode': 'mean_abs', 'trainer:__choice__': 'ForecastingStandardTrainer', 'network_embedding:__choice__': 'NoEmbedding', 'data_loader:window_size': 2, 'loss:__choice__': 'DistributionLoss', 'loss:DistributionLoss:dist_cls': 'studentT', 'loss:DistributionLoss:forecast_strategy': 'sample', 'loss:DistributionLoss:aggregation': 'median', 'loss:DistributionLoss:num_samples': 100, 'network_backbone:__choice__': 'flat_encoder', 'network_backbone:flat_encoder:__choice__': 'MLPEncoder', 'network_backbone:flat_encoder:MLPEncoder:num_groups': 1, 'network_backbone:flat_encoder:MLPEncoder:num_units_1': 40, 'network_backbone:flat_encoder:MLPEncoder:activation': 'relu', 'network_backbone:flat_encoder:MLPEncoder:use_dropout': False, 'network_backbone:flat_encoder:MLPEncoder:normalization': 'NoNorm', 'network_backbone:flat_encoder:MLPDecoder:num_layers': 0, 'network_backbone:flat_encoder:MLPDecoder:has_local_layer': True, 'network_backbone:flat_encoder:MLPDecoder:units_local_layer': 40, 'feature_encoding:__choice__': 'OneHotEncoder', 'scaler:scaling_mode': 'standard'} into a Configuration with error Trying to set illegal value 'OneHotEncoder' (type '<class 'str'>') for hyperparameter 'feature_encoding:__choice__, Type: Categorical, Choices: {NoEncoder}, Default: NoEncoder' (default-value has type '<class 'str'>').. Therefore, it can't be used as an initial configuration as it does not match the current config space.
  warnings.warn(f"Failed to convert {configuration_dict} into"

Your Local Environment

Operating System, version Ubuntu 20.04
Python, version Python 3.8
Outputs of pip freeze or conda list pytorch=1.12+cu116

dengdifan commented 1 year ago

Thanks for the reporting! We will add another function to change the illegal values to the default configuration values and add it to the next release ASAP.

ravinkohli commented 1 year ago

Thanks for the reporting! We will add another function to change the illegal values to the default configuration values and add it to the next release ASAP.

I think it would not be included in the next release as that is coming soon with bug fixes. We'll take a look at this in the next major release.

8W9aG commented 1 year ago

I'm currently seeing this issue as well, is there a workaround I can use right now?