Open BalyshevArtem opened 3 weeks ago
Result of the tool from https://github.com/Samsung/ONE/pull/13479 for our target model: Default training - training all network, all operations, and all tensors:
CROSS_ENTROPY_METRIC_RESULT = 0.187580
PEAK_MEMORY_ESTIMATION_RESULT = 148 624 bytes
Training with training config file, produced by training configure tool:
CROSS_ENTROPY_METRIC_RESULT = 0.187278
PEAK_MEMORY_ESTIMATION_RESULT = 62 160 bytes
Let's introduce training configure tool. This tool should find a configuration for network training with some optimal memory costs. The Training Tool will be able to perform actions to improve the training process:
Related issue: https://github.com/Samsung/ONE/issues/12873