Some confusion with "cash_cocktail" option

[vlawhern]

I ran into your paper not too long ago and found it pretty interesting, thanks for sharing the code.

I'm looking to run this on my own dataset and I'm a bit confused as to the cash_cocktail option in main_experiment.py. My impression is that this automatically turns on all the options to search for HPO but when I run the code I get output that looks like this:

{'task_id': 233088, 'wall_time': 9000, 'func_eval_time': 1000, 'epochs': 105, 'seed': 11, 'tmp_dir': './runs/autoPyTorch_cocktails', 'output_dir': './runs/autoPyTorch_cocktails', 'nr_workers': 6, 'nr_threads': 1, 'cash_cocktail': True, 'use_swa': [False], 'use_se': [False], 'use_lookahead': [False], 'use_weight_decay': [False], 'use_batch_normalization': [False], 'use_skip_connection': [False], 'use_dropout': [False], 'mb_choice': 'none', 'augmentation': 'standard'}
{'task_id': 233088, 'wall_time': 9000, 'func_eval_time': 1000, 'epochs': 105, 'seed': 11, 'tmp_dir': './runs/autoPyTorch_cocktails', 'output_dir': './runs/autoPyTorch_cocktails', 'nr_workers': 6, 'nr_threads': 1, 'cash_cocktail': True, 'use_swa': [False], 'use_se': [False], 'use_lookahead': [False], 'use_weight_decay': [False], 'use_batch_normalization': [False], 'use_skip_connection': [False], 'use_dropout': [False], 'mb_choice': 'none', 'augmentation': 'standard'}

It looks like the options aren't being used? e.g. 'use_swa': [False], 'use_se': [False], 'use_lookahead': [False],...

I guess to rephrase the question: if one were to use your code on their private dataset, what options do you need to pass in to ensure that you are doing the full HPO? is it just the cash_cocktail flag?

[ArlindKadra]

Hello @vlawhern,

I am glad you found the work interesting :). You are correct in your understanding, you would only need to use the cash_cocktail flag. When that option is triggered everything else related to the updates for individual ingredients is ignored: https://github.com/releaunifreiburg/WellTunedSimpleNets/blob/494c978958d1c25e347e0f744c6a26a483734a41/utilities.py#L333

What you are seeing are the default values given for the arguments. When the cash cocktail option is not enabled, you could control the activation of the different ingredients by the arguments.

Let me know if anything else is unclear :)