search

openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
406 stars 135 forks source link

Can't launch FEDOT framework in Linux (Ubuntu 20.04.3 LTS) #652

Open dmitryglhf opened 2 days ago

dmitryglhf commented 2 days ago

Description Unable to launch the FEDOT framework using the automlbenchmark repository in Linux. The process fails despite sufficient resources and prior troubleshooting efforts. Works fine on Windows 10 using Docker.

Environment Details

Steps to Reproduce

  1. Clone the automlbenchmark repository:
git clone https://github.com/openml/automlbenchmark.git --branch stable --depth 1
cd automlbenchmark
  1. Create and activate a virtual environment:
python3.9 -m venv venv
source venv/bin/activate
  1. Install dependencies:
python3.9 -m pip install --upgrade pip
python3.9 -m pip install -r requirements.txt
  1. Run the FEDOT benchmark:
python3.9 runbenchmark.py fedot

Observations

Logs 

Setup of framework FEDOT completed successfully.

-----------------------------------------
Starting job local.test.test.kc2.0.FEDOT.
Assigning 2 cores (total=12) for new task kc2.
[MONITORING] [python3.9 [176658]] CPU Utilization: 4.5%
Assigning 21060 MB (total=24047 MB) for new kc2 task.
[MONITORING] [python3.9 [176658]] Memory Usage: 3.9%
[MONITORING] [python3.9 [176658]] Disk Usage: 76.4%
Running task kc2 on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'classification', 'name': 'kc2', 'openml_task_id': 3913, 'test_server': False, 'fold': 0, 'metric': 'auc', 'metrics': ['auc', 'logloss', 'acc', 'balacc'], 'seed': 1870253876, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21060, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/0/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'binary', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/0/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/0/metadata.json`.
Metric scores: { 'acc': nan,
  'app_version': '2.1.7',
  'auc': nan,
  'balacc': nan,
  'constraint': 'test',
  'duration': nan,
  'fold': 0,
  'framework': 'FEDOT',
  'id': 'openml.org/t/3913',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'logloss': nan,
  'metric': 'auc',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': 1870253876,
  'task': 'kc2',
  'training_duration': nan,
  'type': 'binary',
  'utc': '2024-11-25T12:15:36',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.kc2.0.FEDOT` executed in 0.763 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.

-----------------------------------------
Starting job local.test.test.kc2.1.FEDOT.
Assigning 2 cores (total=12) for new task kc2.
Assigning 21057 MB (total=24047 MB) for new kc2 task.
Running task kc2 on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'classification', 'name': 'kc2', 'openml_task_id': 3913, 'test_server': False, 'fold': 1, 'metric': 'auc', 'metrics': ['auc', 'logloss', 'acc', 'balacc'], 'seed': 1870253877, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21057, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/1/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'binary', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/1/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/kc2/1/metadata.json`.
Metric scores: { 'acc': nan,
  'app_version': '2.1.7',
  'auc': nan,
  'balacc': nan,
  'constraint': 'test',
  'duration': nan,
  'fold': 1,
  'framework': 'FEDOT',
  'id': 'openml.org/t/3913',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'logloss': nan,
  'metric': 'auc',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': 1870253877,
  'task': 'kc2',
  'training_duration': nan,
  'type': 'binary',
  'utc': '2024-11-25T12:15:37',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.kc2.1.FEDOT` executed in 0.316 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.

------------------------------------------
Starting job local.test.test.iris.0.FEDOT.
Assigning 2 cores (total=12) for new task iris.
Assigning 21056 MB (total=24047 MB) for new iris task.
Running task iris on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'classification', 'name': 'iris', 'openml_task_id': 59, 'test_server': False, 'fold': 0, 'metric': 'logloss', 'metrics': ['logloss', 'acc', 'balacc'], 'seed': 1870253876, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21056, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/0/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 
'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'multiclass', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/0/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/0/metadata.json`.
Metric scores: { 'acc': nan,
  'app_version': '2.1.7',
  'balacc': nan,
  'constraint': 'test',
  'duration': nan,
  'fold': 0,
  'framework': 'FEDOT',
  'id': 'openml.org/t/59',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'logloss': nan,
  'metric': 'neg_logloss',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': 1870253876,
  'task': 'iris',
  'training_duration': nan,
  'type': 'multiclass',
  'utc': '2024-11-25T12:15:37',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.iris.0.FEDOT` executed in 0.275 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.

------------------------------------------
Starting job local.test.test.iris.1.FEDOT.
Assigning 2 cores (total=12) for new task iris.
Assigning 21055 MB (total=24047 MB) for new iris task.
Running task iris on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'classification', 'name': 'iris', 'openml_task_id': 59, 'test_server': False, 'fold': 1, 'metric': 'logloss', 'metrics': ['logloss', 'acc', 'balacc'], 'seed': 1870253877, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21055, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/1/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 
'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'multiclass', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/1/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/iris/1/metadata.json`.
Metric scores: { 'acc': nan,
  'app_version': '2.1.7',
  'balacc': nan,
  'constraint': 'test',
  'duration': nan,
  'fold': 1,
  'framework': 'FEDOT',
  'id': 'openml.org/t/59',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'logloss': nan,
  'metric': 'neg_logloss',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'result': nan,
  'seed': 1870253877,
  'task': 'iris',
  'training_duration': nan,
  'type': 'multiclass',
  'utc': '2024-11-25T12:15:37',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.iris.1.FEDOT` executed in 0.272 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.

-------------------------------------------------
Starting job local.test.test.cholesterol.0.FEDOT.
Assigning 2 cores (total=12) for new task cholesterol.
Assigning 21054 MB (total=24047 MB) for new cholesterol task.
Running task cholesterol on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'regression', 'name': 'cholesterol', 'openml_task_id': 2295, 'test_server': False, 'fold': 0, 'metric': 'rmse', 'metrics': ['rmse', 'r2', 'mae'], 'seed': 1870253876, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21054, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/0/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'regression', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/0/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/0/metadata.json`.
Metric scores: { 'app_version': '2.1.7',
  'constraint': 'test',
  'duration': nan,
  'fold': 0,
  'framework': 'FEDOT',
  'id': 'openml.org/t/2295',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'mae': nan,
  'metric': 'neg_rmse',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'r2': nan,
  'result': nan,
  'rmse': nan,
  'seed': 1870253876,
  'task': 'cholesterol',
  'training_duration': nan,
  'type': 'regression',
  'utc': '2024-11-25T12:15:38',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.cholesterol.0.FEDOT` executed in 0.303 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.

-------------------------------------------------
Starting job local.test.test.cholesterol.1.FEDOT.
Assigning 2 cores (total=12) for new task cholesterol.
Assigning 21054 MB (total=24047 MB) for new cholesterol task.
Running task cholesterol on framework FEDOT with config:
TaskConfig({'framework': 'FEDOT', 'framework_params': {}, 'framework_version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027', 'type': 'regression', 'name': 'cholesterol', 'openml_task_id': 2295, 'test_server': False, 'fold': 1, 'metric': 'rmse', 'metrics': ['rmse', 'r2', 'mae'], 'seed': 1870253877, 'job_timeout_seconds': 120, 'max_runtime_seconds': 60, 'cores': 2, 'max_mem_size_mb': 21054, 'min_vol_size_mb': -1, 'input_dir': '/home/valeryp/.cache/openml', 'output_dir': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945', 'output_predictions_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/1/predictions.csv', 'tag': None, 'command': 'runbenchmark.py fedot', 'git_info': {'repo': 'https://github.com/openml/automlbenchmark.git', 'branch': 'HEAD', 'commit': '73823c55d8674761aab25520a75cbeea065ca124', 'tags': ['stable', 'v2.1.7'], 'status': ['## HEAD (no branch)', ' M frameworks/FEDOT/setup.sh']}, 'measure_inference_time': False, 'ext': {}, 'quantile_levels': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], 'type_': 'regression', 'output_metadata_file': '/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/1/metadata.json'})
Running cmd `/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py`
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py", line 5, in <module>
    from fedot.api.main import Fedot

ModuleNotFoundError: No module named 'fedot'

Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/essdata/00_glhf/automlbenchmark/amlb/benchmark.py", line 578, in run
    meta_result = self.benchmark.framework_module.run(self._dataset, task_config)
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/__init__.py", line 24, in run
    return run_in_venv(__file__, "exec.py",
  File "/var/essdata/00_glhf/automlbenchmark/frameworks/shared/caller.py", line 134, in run_in_venv
    output, err = run_cmd(cmd, *args,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 281, in run_cmd
    raise e
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 255, in run_cmd
    completed = run_subprocess(str_cmd if params.shell else full_cmd,
  File "/var/essdata/00_glhf/automlbenchmark/amlb/utils/process.py", line 98, in run_subprocess
    raise subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit status 1.
Loading metadata from `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/predictions/cholesterol/1/metadata.json`.
Metric scores: { 'app_version': '2.1.7',
  'constraint': 'test',
  'duration': nan,
  'fold': 1,
  'framework': 'FEDOT',
  'id': 'openml.org/t/2295',
  'info': 'CalledProcessError: Command '
          "'/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python "
          '-W ignore '
          "/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' "
          'returned non-zero exit stat…',
  'mae': nan,
  'metric': 'neg_rmse',
  'mode': 'local',
  'models_count': nan,
  'params': '',
  'predict_duration': nan,
  'r2': nan,
  'result': nan,
  'rmse': nan,
  'seed': 1870253877,
  'task': 'cholesterol',
  'training_duration': nan,
  'type': 'regression',
  'utc': '2024-11-25T12:15:38',
  'version': '#b0618dfa20cf43bc5bf1c08f27f054a0cecd6027'}
Job `local.test.test.cholesterol.1.FEDOT` executed in 0.302 seconds.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/fedot.test.test.local.20241125T115945/scores/results.csv`.
Scores saved to `/var/essdata/00_glhf/automlbenchmark/results/results.csv`.
All jobs executed in 3.253 seconds.
[MONITORING] [python3.9 [176658]] CPU Utilization: 7.9%
[MONITORING] [python3.9 [176658]] Memory Usage: 3.9%
[MONITORING] [python3.9 [176658]] Disk Usage: 76.4%
Processing results for fedot.test.test.local.20241125T115945
Summing up scores for current run:
               id        task  fold framework constraint      metric  duration       seed
                                                                                                                info
openml.org/t/3913         kc2     0     FEDOT       test         auc       0.8 1870253876 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…
openml.org/t/3913         kc2     1     FEDOT       test         auc       0.3 1870253877 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…
  openml.org/t/59        iris     0     FEDOT       test neg_logloss       0.3 1870253876 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…
  openml.org/t/59        iris     1     FEDOT       test neg_logloss       0.3 1870253877 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…
openml.org/t/2295 cholesterol     0     FEDOT       test    neg_rmse       0.3 1870253876 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…
openml.org/t/2295 cholesterol     1     FEDOT       test    neg_rmse       0.3 1870253877 CalledProcessError: Command '/var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/venv/bin/python -W ignore /var/essdata/00_glhf/automlbenchmark/frameworks/FEDOT/exec.py' returned non-zero exit stat…

Questions

  1. Are there any specific dependencies or configurations required for FEDOT on Linux?

  2. Could the issue be related to differences in the Python environment between Linux and Docker setups?

  3. What additional debugging steps or environment checks should be performed?

Thank you for your assistance!

dmitryglhf commented 2 days ago

I apologize for my inattention ModuleNotFoundError: No module named 'fedot' is cause of problem. It worked to install this package in FEDOT/venv directly. I thought I had tried this method before.

PGijsbers commented 2 days ago

The benchmark should be able to install the package for you, though. I'll re-open until I have the chance to see if it is reproducible.