slds-lmu / yahpo_gym

Surrogate benchmarks for HPO problems
26 stars 3 forks source link

KeyError: 'q' #79

Open Myrmustin opened 1 year ago

Myrmustin commented 1 year ago

There seems to be an issue with running yahpo.

I run the following code:

from yahpo_gym import *
from yahpo_gym import local_config, benchmark_set

def main(): 

    local_config.init_config()
    local_config.set_data_path("yahpo_data")

    b = benchmark_set.BenchmarkSet(scenario="lcbench")

and the error traceback is the following:

Traceback (most recent call last):
  File "/Users/ross/Desktop/AutoML/mainTwo.py", line 15, in <module>
    main()
  File "/Users/ross/Desktop/AutoML/mainTwo.py", line 11, in main
    b = benchmark_set.BenchmarkSet(scenario="lcbench", instance= 126025)
  File "/Users/ross/.local/share/virtualenvs/AutoML-nNY38BET/lib/python3.10/site-packages/yahpo_gym/benchmark_set.py", line 50, in __init__
    self.config_space = self._get_config_space()
  File "/Users/ross/.local/share/virtualenvs/AutoML-nNY38BET/lib/python3.10/site-packages/yahpo_gym/benchmark_set.py", line 371, in _get_config_space
    cs = CS_json.read(json_string)
  File "/Users/ross/.local/share/virtualenvs/AutoML-nNY38BET/lib/python3.10/site-packages/ConfigSpace/read_and_write/json.py", line 444, in read
    _construct_hyperparameter(
  File "/Users/ross/.local/share/virtualenvs/AutoML-nNY38BET/lib/python3.10/site-packages/ConfigSpace/read_and_write/json.py", line 524, in _construct_hyperparameter
    q=hyperparameter["q"],
KeyError: 'q'

I have verified that the data folder has the appropriate structure and followed the instructions to install yapoGym. Used python 3.9 and 3.10 with the same error on a mac. Any feedback would be highly appreciated!

sumny commented 1 year ago

Hi @Myrmustin, thanks for the issue - can reproduce it.

it seems that the new ConfigSpace 0.7.1 (released two weeks ago) conflicts with our serialized configspaces in yahpo_data. For now, making sure that you have ConfigSpace 0.6.1 installed should resolve it.

pip install ConfigSpace==0.6.1

I'll make sure to explicitly require <= 0.6.1 and document this.

Let me know if you stumble across other issues.