TUM-DAML / seml

SEML: Slurm Experiment Management Library
Other
165 stars 29 forks source link

adding wildcard feature #100

Closed yascho closed 1 year ago

yascho commented 1 year ago

Reference issue

Resolve #99

What does this implement/fix?

Support for adding multiple configs at once, e.g.:

seml experiment add config0.yaml config1.yaml
seml experiment add *.yaml
n-gao commented 1 year ago

I checked out the code and tried it in the example directory and got the following:

seml/examples$ seml seml_test add *.yaml
Traceback (most recent call last):
  File "/nfs/staff-ssd/gaoni/miniconda3/envs/seml_test/bin/seml", line 33, in <module>
    sys.exit(load_entry_point('seml', 'console_scripts', 'seml')())
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/main.py", line 298, in main
    f(**vars(command))
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/add.py", line 113, in add_configs
    add_experiments(db_collection_name, config_file, force_duplicates,
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/add.py", line 213, in add_experiments
    add_configs(collection, seml_config, slurm_config, configs, uploaded_files, git_info)
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/add.py", line 113, in add_configs
    add_experiments(db_collection_name, config_file, force_duplicates,
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/add.py", line 136, in add_experiments
    seml_config, slurm_config, experiment_config = read_config(config_file)
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/config.py", line 372, in read_config
    with open(config_path, 'r') as conf:
FileNotFoundError: [Errno 2] No such file or directory: '/nfs/staff-ssd/gaoni/repos/seml_scholten/executable'
yascho commented 1 year ago

Hi, should work now.

Further fixes:

n-gao commented 1 year ago

Source code reloading does not work properly because the function signatures changed:

Traceback (most recent call last):
  File "/nfs/staff-ssd/gaoni/miniconda3/envs/seml_test/bin/seml", line 33, in <module>
    sys.exit(load_entry_point('seml', 'console_scripts', 'seml')())
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/main.py", line 298, in main
    f(**vars(command))
  File "/nfs/staff-ssd/gaoni/repos/seml_scholten/seml/manage.py", line 445, in reload_sources
    check_config(seml_config['executable'], seml_config['conda_environment'], configs)
TypeError: check_config() missing 1 required positional argument: 'working_dir'

Do we still need the explicit working_directory in the reload_sources function if we pass it downstream?

n-gao commented 1 year ago

LGTM, thank you! Closes #99