Calculate total_steps for a scheduler dynamically.
Motivation
Currently, a scheduler's total_steps must be specified manually. Each time the network topology changes, we need to adapt the value.
For example, I used 2 GPUs during debugging, but within the tests, I wanted to use only a single. Manually adapting the config is then needed, which is somewhat hacky.
Feature request
Calculate total_steps for a scheduler dynamically.
Motivation
Currently, a scheduler's total_steps must be specified manually. Each time the network topology changes, we need to adapt the value. For example, I used 2 GPUs during debugging, but within the tests, I wanted to use only a single. Manually adapting the config is then needed, which is somewhat hacky.