Support all the softmax extensions and cherry-pick transformer-related commits

[hubertlu-tw]

• Cherry-picked transformer related commits from the upstream
• Support generic_scaled_masked_softmax_cuda
• Support scaled_softmax_cuda
• Support fused_weight_gradient_mlp_cuda for ROCm
• Add optimizers, clip_grad in run_rocm_extensions.py
• Fix the bug in run_rocm_extensions.py To run the extension unit tests, please follow the following commands:

  cd apex/contrib/test
  APEX_TEST_WITH_ROCM=1 APEX_SKIP_FLAKY_TEST=1 python3 run_rocm_extensions.py

To run the transformer unit tests, please follow the following commands:

python tests/L0/run_test.py --include run_transformer

Ran 120 tests in 506.928s FAILED (errors=7, skipped=55)

TODO: We will need to work on an IFU PR and start to look into the failed tests in order to skip them on ROCm.