Adding user keys to config

[oliver-lloyd]

Hi,

I am trying to add a simple bool flag to the default config that can be retrieved and set as an attribute in lookup_embedder.py. I have added the following to the end of config-default.yaml:

user:
  +++: +++
  my_bool: False

and the following to my job config (toy-complex-search-ax.yaml):

user:
  my_bool: True

with this line added in LookupEmbedder.init(): self.my_bool = self.get_option("user.my_bool")

Trying to run the job with these modifications causes an error (traceback below). Could someone please explain why this exception occurs, and how I can properly set custom flags in job configs?

Many thanks in advance!

Failed to create embedder lookup_embedder (class LookupEmbedder).
[77766258] Failed to create model complex (class ComplEx).
[497d4ea3] Trial 00000 failed: KeyError('Error accessing user for key complex.entity_embedder.user.my_bool')
[497d4ea3] Aborting search due to failure of trial 00000
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 56, in get
    result = result[name]
KeyError: 'type'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 93, in get_default
    parent_type = self.get(parent + "." + "type")
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 58, in get
    raise KeyError(f"Error accessing {name} for key {key}")
KeyError: 'Error accessing type for key lookup_embedder.type'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/fu19841/miniconda3/envs/libkge/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/search.py", line 232, in _run_train_job
    raise e
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/search.py", line 143, in _run_train_job
    parent_job=search_job,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/job.py", line 83, in create
    config, dataset, parent_job=parent_job, model=model
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/train.py", line 136, in create
    forward_only=forward_only,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/misc.py", line 38, in init_from
    return getattr(module, class_name)(*args, **kwargs)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/train_KvsAll.py", line 32, in __init__
    config, dataset, parent_job, model=model, forward_only=forward_only
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/train.py", line 71, in __init__
    self.model: KgeModel = KgeModel.create(config, dataset)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/model/kge_model.py", line 497, in create
    init_for_load_only=init_for_load_only,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/misc.py", line 38, in init_from
    return getattr(module, class_name)(*args, **kwargs)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/model/complex.py", line 61, in __init__
    init_for_load_only=init_for_load_only,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/model/kge_model.py", line 388, in __init__
    init_for_load_only=init_for_load_only,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/model/kge_model.py", line 281, in create
    init_for_load_only=init_for_load_only,
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/misc.py", line 38, in init_from
    return getattr(module, class_name)(*args, **kwargs)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/model/embedder/lookup_embedder.py", line 32, in __init__
    self.my_bool = self.get_option("user.my_bool")
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 619, in get_option
    return self.config.get_default(self.configuration_key + "." + name)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 103, in get_default
    raise e
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 83, in get_default
    return self.get(key)
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/config.py", line 58, in get
    raise KeyError(f"Error accessing {name} for key {key}")
KeyError: 'Error accessing user for key complex.entity_embedder.user.my_bool'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/fu19841/miniconda3/envs/libkge/bin/kge", line 33, in <module>
    sys.exit(load_entry_point('libkge', 'console_scripts', 'kge')())
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/cli.py", line 285, in main
    job.run()
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/job.py", line 159, in run
    result = self._run()
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/search_auto.py", line 162, in _run
    (self, trial_no, config, self.num_trials, list(parameters.keys())),
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/search.py", line 75, in submit_task
    self.wait_task()
  File "/Users/fu19841/Documents/thesis_analysis/kge/kge/job/search.py", line 97, in wait_task
    self.ready_task_results.append(task.result())
  File "/Users/fu19841/miniconda3/envs/libkge/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/Users/fu19841/miniconda3/envs/libkge/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
KeyError: 'Error accessing user for key complex.entity_embedder.user.my_bool'

[rgemulla]

self.get_option retrieves an option under the key for the current class (here: complex.entity_embedder), which is not what you want. Use self.config.get instead.

[oliver-lloyd]

self.get_option retrieves an option under the key for the current class (here: complex.entity_embedder), which is not what you want. Use self.config.get instead.

Brilliant, thank you!