Fix Namespace error

[Innixma]

469 appears to have introduced a bug where running in AWS mode will crash immediately due to a key error if the user doesn't specify instance_tags in a custom config. This code fixes the error.

Command:

python runbenchmark.py constantpredictor test -f 0 -m aws -p 10

Error:

---------------------------------------------------
Starting job aws.test.test.kc2.0.constantpredictor.
Starting new EC2 instance with params: constantpredictor test test -t kc2 -f 0 -Xseed=380416698
Job aws.test.test.kc2.0.constantpredictor failed with: 'Namespace' object has no attribute 'instance_tags'
fatal: not a git repository (or any of the parent directories): .git

Metric scores: { 'app_version': 'dev [NA, NA, NA]',
  'constraint': 'test',
  'duration': nan,
  'fold': 0,
  'framework': 'constantpredictor',
  'id': 'openml.org/t/3913',
  'info': "AttributeError: 'Namespace' object has no attribute 'instance_tags'",
  'metric': None,
  'mode': 'aws',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': None,
  'task': 'kc2',
  'training_duration': nan,
  'type': None,
  'utc': '2022-08-04T00:46:26',
  'version': None}
Job `aws.test.test.kc2.0.constantpredictor` executed in 0.021 seconds.
Error when handling state change to State.completing for job `aws.test.test.kc2.0.constantpredictor`: cannot unpack non-iterable bool object
Traceback (most recent call last):
  File "/home/ubuntu/benchmark/ag-2022_08_03_hstree/tmp6/automlbenchmark/amlb/job.py", line 158, in set_state
    skip_default = bool(self._on_state(state))
  File "/home/ubuntu/benchmark/ag-2022_08_03_hstree/tmp6/automlbenchmark/amlb/runners/aws.py", line 388, in _on_state
    terminate, failure = self._download_results(_self.ext.instance_id)
TypeError: cannot unpack non-iterable bool object

[sebhrusen]

@Innixma I can't reproduce your error in a default environment: with unmodified resources/config.yaml and with custom config without instance_tags entry.

The only way for me to reproduce the error is to add the following to the custom config.yaml:

aws:
  ec2:
    instance_tags:

which will override the default instance_tags entry and set it to None instead of {}, leading to the following error, slightly different from yours:

Starting new EC2 instance with params: constantpredictor test test -t kc2 -f 0 -Xseed=1
Job aws.test.test.kc2.0.constantpredictor failed with: unsupported operand type(s) for |: 'NoneType' and 'Namespace'
Metric scores: { 'app_version': 'dev [master, 88186b3]',
  'constraint': 'test',
  'duration': nan,
  'fold': 0,
  'framework': 'constantpredictor',
  'id': 'openml.org/t/3913',
  'info': "TypeError: unsupported operand type(s) for |: 'NoneType' and "
          "'Namespace'",
  'metric': None,
  'mode': 'aws',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': None,
  'task': 'kc2',
  'training_duration': nan,
  'type': None,
  'utc': '2022-08-04T15:32:53',
  'version': None}
Job `aws.test.test.kc2.0.constantpredictor` executed in 0.218 seconds.
Error when handling state change to State.completing for job `aws.test.test.kc2.0.constantpredictor`: cannot unpack non-iterable bool object
Traceback (most recent call last):
  File "/Users/seb/repos/ml/automlbenchmark/amlb/job.py", line 158, in set_state
    skip_default = bool(self._on_state(state))
  File "/Users/seb/repos/ml/automlbenchmark/amlb/runners/aws.py", line 388, in _on_state
    terminate, failure = self._download_results(_self.ext.instance_id)
TypeError: cannot unpack non-iterable bool object

But otherwise, due to the hierarchical structure of the configuration files, this entry is an empty map, not None.

Did you by any chance edit the resources/config.yaml manually instead? Commenting out the instance_tags entry in that file is the only way for me to reproduce this.