adaptiveScheduler methods

[dmccloskey]

Description

Methods to allow for easy implementation of complex training schedules that involve adapting the population size, # of "mutations", etc., based on the network performance. In addition, it would be necessary to have methods to increase or decrease the difficulty of the task depending upon the network performance.

Objectives

• [x] model change scheduling mechanism for ModelReplicator (i.e. update of setRandomModifications)
• [x] model training hyper parameter mechanisms for ModelTrainer (i.e., updates of n_epochs, solver parameters, time-steps during back propogation, etc.)
• [x] population training hyper parameter mechanisms for PopulationTrainer (i.e., n_top, n_random, and n_replicates_per_model)

[dmccloskey]

Notes 2018-08-05

Population dynamic triggers

• generations
• total error of selected models
• total error of selected models slope

Model modifications triggers

• generations
• epochs
• model error
• model error slope
• model size

Training modifications triggers

• generations
• epochs
• model size
• model error
• model slope

[dmccloskey]

Notes 2018-08-05

Population dynamic changes

• n_top
• n_random
• n_replicates
• n_generations

Model modifications triggers

• random modification parameters

Training modifications triggers

• n_epochs_training
• n_epochs_validation
• FP/BP/CE memory steps
• memory_size
• batch_size