In order to allow for changing experiment config parameters from the command command line I've updated ExperimentConfig so that, by default, all abstract methods are instance-level rather than class level. This change is backwards compatible with all the code out there (as class methods can always be called whenever instance methods can be). This now means that we can change the behavior of these methods by passing different values to the experiment config's initializer. To pass these values via the command line, I've added a --config_kwargs option which allows people to pass key word arguments from the command line (e.g. --config_kwargs "{'num_train_processes': 4}").
I'm of the opinion that the way we ran testing from the command line was too "magical" as it basically made use of a lot of internal directory naming conventions. I've deprecated a bunch of this to make it a bit more manual/simple, basically people now need to directly specify checkpoints and don't have to think quite as much about test dates.
jordis-ai2
commented
3 years ago
This PR implements two changes:
ExperimentConfigso that, by default, all abstract methods are instance-level rather than class level. This change is backwards compatible with all the code out there (as class methods can always be called whenever instance methods can be). This now means that we can change the behavior of these methods by passing different values to the experiment config's initializer. To pass these values via the command line, I've added a--config_kwargsoption which allows people to pass key word arguments from the command line (e.g.--config_kwargs "{'num_train_processes': 4}").This solves #201 .