Add scheduler to run instances

[panahiparham]

The Scheduler

This class is responsible for preparing instances of an experiment for execution.

Inputs

1. Experiment name string
2. List of integer seeds

Functionality

1. Generate list of tuples (part_name, config_id, seed) that fully specifies an instance of an experiment
2. Handle running instances across many cores in one system.
3. Handle running instances on a cluster with GNU parallel

Proposed interface

class Scheduler:
    def __init__(self, exp_name: str, seeds: List[int], version: Maybe[int] = None, base: Maybe[str] = None):
         pass

    ''' iterate over all instances to be run checking pre-existing results'''
    def enumerate_runs(self) -> Iterable[Tuple[str, int, int]]:
        pass

Assumes access to results_exist(name: str, config: str, seed: str)

[panahiparham]

@andnp

• What do you think of the proposed functionalities for the Scheduler class?
• Should functionalities 2 and 3 be here?
• Is there anything I missed?

[panahiparham]

• Use python's Pool instead of GNU parallel
• Public API should not have Maybe
• Use group utility to break into jobs
• results_exist
  • Require user to subclass
  • Pass to enumerate_runs
  • (*) Builder approach

[panahiparham]

Checking for existing results can be slow, we can parallelize reading from database.