Closed ioana-blue closed 5 years ago
Some pieces of the code are not friendly to hyperparameter tuning. For example, there is no way to do no LR decay the way the code is written now:
i.e., if args.lr_decay_rate is not None
if args.lr_decay_rate is not None
will never evaluate to true because lr_decay_rate has a default value. This line should probably be replace with something like:
lr_decay_rate
if args.lr_decay_rate
which allows the lr_decay_rate to be set to 0 and that if portion doesn't execute.
Some pieces of the code are not friendly to hyperparameter tuning. For example, there is no way to do no LR decay the way the code is written now:
i.e.,
if args.lr_decay_rate is not Nonewill never evaluate to true because
lr_decay_ratehas a default value. This line should probably be replace with something like:if args.lr_decay_ratewhich allows the
lr_decay_rateto be set to 0 and that if portion doesn't execute.