As detailed in the titles, this PR refines the API of BasicOptimizer and add more checks for use_distributed_optimizer settings. In this PR, we also add a test case about shared weight on single machine. We will later release an integration test when our PP is ready.
As detailed in the titles, this PR refines the API of BasicOptimizer and add more checks for
use_distributed_optimizer
settings. In this PR, we also add a test case about shared weight on single machine. We will later release an integration test when our PP is ready.