An error about Hydra when running examples/few-shot-classification

[JiaxiLi001]

Sorry but I'm just a novice in hydra. I just simply ran the first script in few-shot-classification/README.md, and the error occurred as below, thanks for your help. Traceback (most recent call last): File "/data/lijiaxi-slurm/Research/IRlab/rl-prompt/examples/few-shot-classification/run_fsc.py", line 53, in main() File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/main.py", line 90, in decorated_main _run_hydra( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra _run_app( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 452, in _run_app run_and_report( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 216, in run_and_report raise ex File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 213, in run_and_report return func() File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 453, in lambda: hydra.run( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 105, in run cfg = self.compose_config( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 594, in compose_config cfg = self.config_loader.load_configuration( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration return self._load_configuration_impl( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 233, in _load_configuration_impl parsed_overrides = parser.parse_overrides(overrides=overrides) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/core/override_parser/overrides_parser.py", line 96, in parse_overrides raise OverrideParseException( hydra.errors.OverrideParseException: no viable alternative at input '[sst-2,' See https://hydra.cc/docs/next/advanced/override_grammar/basic for details

[JiaxiLi001]

It seems like there can't exist any whitespace in the list of the script in fsc/README.md, but another error occurred as below: Traceback (most recent call last): File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 379, in _apply_overrides_to_config OmegaConf.update(cfg, key, value, merge=True) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 741, in update root.setattr(last_key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 338, in setattr self._format_and_raise(key=key, value=value, cause=e) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/base.py", line 231, in _format_and_raise format_and_raise( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/_utils.py", line 899, in format_and_raise _raise(ex, cause) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/_utils.py", line 797, in _raise raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 334, in setattr self.set_impl(key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl self._set_item_impl(key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 604, in _set_item_impl self.dict__["_content"][key]._set_value(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 46, in _set_value self._val = self.validate_and_convert(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 76, in validate_and_convert return self._validate_and_convert_impl(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 194, in _validate_and_convert_impl raise ValidationError("Cannot convert '$VALUE_TYPE' to string: '$VALUE'") omegaconf.errors.ValidationError: Cannot convert 'list' to string: '['sst-2', 'yelp-2', 'mr', 'cr', 'agnews', 'sst-5', 'yelp-5']' full_key: dataset object_type=Config

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data/lijiaxi-slurm/Research/IRlab/rl-prompt/examples/few-shot-classification/run_fsc.py", line 53, in main() File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/main.py", line 90, in decorated_main _run_hydra( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra _run_app( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 452, in _run_app run_and_report( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 216, in run_and_report raise ex File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 213, in run_and_report return func() File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/utils.py", line 453, in lambda: hydra.run( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 105, in run cfg = self.compose_config( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 594, in compose_config cfg = self.config_loader.load_configuration( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration return self._load_configuration_impl( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 265, in _load_configuration_impl ConfigLoaderImpl._apply_overrides_to_config(config_overrides, cfg) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 386, in _apply_overrides_to_config raise ConfigCompositionException( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 379, in _apply_overrides_to_config OmegaConf.update(cfg, key, value, merge=True) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 741, in update root.setattr(last_key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 338, in setattr self._format_and_raise(key=key, value=value, cause=e) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/base.py", line 231, in _format_and_raise format_and_raise( File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/_utils.py", line 899, in format_and_raise _raise(ex, cause) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/_utils.py", line 797, in _raise raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 334, in setattr self.set_impl(key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl self._set_item_impl(key, value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 604, in _set_item_impl self.dict__["_content"][key]._set_value(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 46, in _set_value self._val = self.validate_and_convert(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 76, in validate_and_convert return self._validate_and_convert_impl(value) File "/data/lijiaxi-slurm/anaconda3/envs/rl-prompt/lib/python3.9/site-packages/omegaconf/nodes.py", line 194, in _validate_and_convert_impl raise ValidationError("Cannot convert '$VALUE_TYPE' to string: '$VALUE'") hydra.errors.ConfigCompositionException: Error merging override dataset=[sst-2,yelp-2,mr,cr,agnews,sst-5,yelp-5]

[mingkaid]

Can you also share the commands you entered for this error to occur? Thanks

[mingkaid]

From the message, it seems that for the dataset parameter, you entered a list instead of a string as is required

[JiaxiLi001]

HYDRA_FULL_ERROR=1 python run_fsc.py \ dataset=[sst-2,yelp-2,mr,cr,agnews,sst-5,yelp-5] \ dataset_seed=[0,1,2,3,4] \ prompt_length=["any integer(optional,default:5)"] \ task_lm=[distilroberta-base,roberta-base,roberta-large,distilgpt2,gpt2,gpt2-medium,gpt2-large,gpt2-xl] \ random_seed=["any integer(optional)"]

[JiaxiLi001]

it seems like I need to remove the whitespace in the list dataset, dataset_seed and task_lm, otherwise it will report the error I first commented in this issue. 🥺thanks for your help.

[mingkaid]

The list items are just options. You need to pick one. One example command will be

python run_fsc.py \
    dataset=sst-2 \
    dataset_seed=0 \
    prompt_length=5 \
    task_lm=roberta-large \
    random_seed=0

Let me know if this works

[JiaxiLi001]

yeah it works, I don't know the list means the options, thanks a lot. And now the problem is how to load the model from huggingface with a server in China🤣I will try to solve it.