Running autotuning fails with `Primary config directory not found.`

lqwk commented 2 years ago

🐛 Bug

When I tried running the auto tuning script, I get an error saying Primary config directory not found. See details below.

To Reproduce

Steps to reproduce the behavior:

Run make install from base directory to install from source
Run python setup.py install from examples/ to install scripts.
Run the following command to run the script

HYDRA_FULL_ERROR=1 python -m llvm_autotuning.tune -m \
  experiment=my-exp \
  outputs=/tmp/logs \
  executor.cpus=32 \
  num_replicas=1 \
  autotuner=nevergrad \
  autotuner.optimization_target=runtime \      
  autotuner.search_time_seconds=600

The full error message is:

Traceback (most recent call last):
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym_examples-0.2.4-py3.8.egg/llvm_autotuning/tune.py", line 37, in <module>
    main()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 375, in _run_hydra
    run_and_report(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 376, in <lambda>
    lambda: hydra.multirun(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 121, in multirun
    cfg = self.compose_config(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 564, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 146, in load_configuration
    return self._load_configuration_impl(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 227, in _load_configuration_impl
    self.ensure_main_config_source_available()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 134, in ensure_main_config_source_available
    self._missing_config_error(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 108, in _missing_config_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config directory not found.
Check that the config directory '/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym_examples-0.2.4-py3.8.egg/llvm_autotuning/config' exists and readable

Expected behavior

The script shouldn't fail.

Environment

Please fill in this checklist:

CompilerGym: latest development branch
How you installed CompilerGym (pip, source): source
OS: MacOS
Python version: 3.7
Build command you used (if compiling from source): make install
GCC/clang version (if compiling from source):
Versions of any other relevant libraries:

You may use the environment collection script to generate most of this information. You can get the script and run it with:

wget https://raw.githubusercontent.com/facebookresearch/CompilerGym/stable/build_tools/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

ChrisCummins commented 2 years ago

Thanks for the detailed report. I repro'd the problem on my end and pushed a fix in #735.

I've added a new test to cover this case and catch repressions.

Small thing - step (2) isn't needed (it's run by make install).

BTW, I see that you are on macOS. Is this an intel Mac or one of the new Apple M-series chips? We don't yet have a build for the Apple M, though you could use docker

Cheers, Chris

lqwk commented 2 years ago

Thanks for the fix! I'm on an intel mac.

facebookresearch / CompilerGym