The list of metadata returned by all benchmarks tuning neural networks should be listed a current list is:
num_parameters: integer value of the number of trainable parameters of the neural network.
budget: scalar value (float/int) of the budget consumed by the neural network. Therefore the budget should be defined for each benchmark (e.g., number of epochs in general).
stopped: boolean value indicating if the evaluation was stopped before consuming the maximum budget.
train_X: scalar value of the training metrics (replace X by the metric name, 1 key per metric).
valid_X: scalar value of the validation metrics (replace X by the metric name, 1 key per metric).
test_X: scalar value of the testing metrics (replace X by the metric name, 1 key per metric).
The list of metadata returned by all benchmarks tuning neural networks should be listed a current list is:
num_parameters
: integer value of the number of trainable parameters of the neural network.budget
: scalar value (float/int) of the budget consumed by the neural network. Therefore the budget should be defined for each benchmark (e.g., number of epochs in general).stopped
: boolean value indicating if the evaluation was stopped before consuming the maximum budget.train_X
: scalar value of the training metrics (replaceX
by the metric name, 1 key per metric).valid_X
: scalar value of the validation metrics (replaceX
by the metric name, 1 key per metric).test_X
: scalar value of the testing metrics (replaceX
by the metric name, 1 key per metric).Also add the
@profile
decorator to all functions.