AI-secure / DecodingTrust

A Comprehensive Assessment of Trustworthiness in GPT Models
https://decodingtrust.github.io/
Creative Commons Attribution Share Alike 4.0 International
264 stars 56 forks source link

MissingMandatoryValue #42

Open richhh520 opened 9 months ago

richhh520 commented 9 months ago

When running: dt-run +toxicity=realtoxicityprompts-toxic \ ++model=hf/llama/llama-2-7b-chat-hf\ ++toxicity.n=25 \ ++toxicity.template=1

The bug is:

sys:1: UserWarning: 'config' is validated against ConfigStore schema with the same name. This behavior is deprecated in Hydra 1.1 and will be removed in Hydra 1.2. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/automatic_schema_matching for migration instructions. /root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/main.py:94: UserWarning: 'config' is validated against ConfigStore schema with the same name. This behavior is deprecated in Hydra 1.1 and will be removed in Hydra 1.2. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/automatic_schema_matching for migration instructions. _run_hydra( [2024-03-07 04:50:41,587][hydra.experimental.callbacks.PickleJobInfoCallback][INFO] - Saving job configs in /root/ruizhe/DecodingTrust/outputs/2024-03-07/04-50-41/.hydra/config.pickle [2024-03-07 04:50:41,588][hydra.experimental.callbacks.PickleJobInfoCallback][INFO] - Saving job_return in /root/ruizhe/DecodingTrust/outputs/2024-03-07/04-50-41/.hydra/job_return.pickle Error executing job with overrides: ['+toxicity=realtoxicityprompts-toxic', '++model=hf/llama/llama-2-7b-chat-hf', '++toxicity.n=25', '++toxicity.template=1'] omegaconf.errors.MissingMandatoryValue: Structured config of type $OBJECT_TYPE has missing mandatory value: $KEY

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/root/miniconda3/envs/dt-test/bin/dt-run", line 8, in sys.exit(main()) File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/_internal/utils.py", line 223, in run_and_report raise ex File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/internal/hydra.py", line 132, in run = ret.return_value File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) File "/root/ruizhe/DecodingTrust/src/dt/main.py", line 32, in main config: BaseConfig = OmegaConf.to_object(raw_config) File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 594, in to_object return OmegaConf.to_container( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 573, in to_container return BaseContainer._to_content( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 288, in _to_content return conf._to_object() File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 747, in _to_object self._format_and_raise( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/base.py", line 231, in _format_and_raise format_and_raise( File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/_utils.py", line 899, in format_and_raise _raise(ex, cause) File "/root/miniconda3/envs/dt-test/lib/python3.9/site-packages/omegaconf/_utils.py", line 797, in _raise raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace omegaconf.errors.MissingMandatoryValue: Structured config of type BaseConfig has missing mandatory value: model_config full_key: model_config object_type=BaseConfig

After adding ++KEY="sk-qH4G4X06lojOFj57nujXXX“, the same bug is also reported.

richhh520 commented 9 months ago

I think its because of BaseConfig is updated.

ujuar001 commented 8 months ago

Were you able to find a workaround @richhh520 ?

richhh520 commented 8 months ago

Yes, it's solved

Were you able to find a workaround @richhh520 ?

liyucheng09 commented 5 months ago

@richhh520 Hi, could you please share how to solve the issue here?

liyucheng09 commented 5 months ago

@ujuar001 Hi, have you found a solution for this issue?